A few days ago, Andreessen Horowitz’s Martin Casado and Matt Bornstein published an interesting piece digging into the world of artificial intelligence (AI) startups, and, more specifically, how those companies perform as businesses. Core to the argument presented is that while founders and investors are wagering “that AI businesses will resemble traditional software companies,” the well-known venture firm is “not so sure.”
Given that TechCrunch cares a lot about startup business fundamentals, the notion that one oft-discussed and well-funded category of venture-backed startup might sport materially less attractive economics than we expected captured our attention.
The Andreessen Horowitz (a16z) perspective is straightforward, arguing that AI-focused companies have lesser gross margins than software companies due to cloud compute and human-input costs, endure issues stemming from “edge-cases” and enjoy less product differentiation from competing companies when compared to software concerns. Today, we’re drilling into the gross margin point, as it’s something inherently numerical that we can get other, informed market participants to weigh in on.
If a16z is correct about AI startups having slimmer gross margins than SaaS companies, they should — all other things held equal — be worth less per dollar of revenue generated; or in simpler terms, they should trade at a revenue multiple discount to SaaS companies, leaving the latter category of technology company still atop the valuation hierarchy.
This matters, given the amount of capital that AI-focused startups have raised.
Is a16z correct about AI gross margins? I wanted to find out. So this week I spoke to a number of investors from firms that have made AI-focused bets to get a handle on their views. Read the full a16z piece, mind. It’s interesting and worth your time.
Today we’re hearing from Rohit Sharma of True Ventures, Jeremy Kaufmann of Scale Venture Partners, Nick Washburn of Intel Capital and Ben Blume of Atomico. We’ll start with a digest of their responses to our questions, with their unedited notes at the end.
AI economics and optimism
We asked our group of venture investors (selected with the help of research from TechCrunch’s Arman Tabatabai) three questions. The first dealt with margins themselves, the second dealt with resulting valuations and, finally, we asked about their current optimism interval regarding AI-focused companies.
Gross margins
Our first question was simple: “Does your experience investing in AI-focused and related companies and startups support the idea that these businesses have gross margins ‘in the 50-60% range’ instead of the higher band often seen by SaaS companies?”
Andreessen Horowitz wrote in its piece that it has seen a “consistent pattern in the financial data of AI companies with gross margins often in the 50-60% range – well below the 60-80%+ benchmark for comparable SaaS businesses.” So, as a starting point, we wanted to test the data. Here’s what we heard back:
Atomico’s Ben Blume backed up the idea to a degree, saying that it’s “true that the compute costs related to AI training and inference are substantial and this does impact an AI company’s gross margin,” and that the “cost of the human in ‘human-in-the-loop’ AI systems also has a clear impact on margin.” But, he also argued that “compute costs have historically trended downwards,” which should help AI-focused companies over time.
Intel Capital’s Nick Washburn told us he agrees “with the early observation that some AI business application software companies run at overall lower gross margins than some traditional non-AI SaaS companies.” He specifically cited “the computational costs of model development and deployment on the AI-side [driving] up COGS,” harming gross margins, as well as the cost of humans. Washburn did, however, differentiate between ” ‘AI services companies’ and ‘AI product companies with a services component,’ ” which can have different economic profiles.
Turning to Scale Venture Partners’ Jeremy Kaufmann, he also agreed, keeping his initial notes tied to earlier-stage startups and saying that “the initial gross margin for many AI companies will absolutely be lower than we see for similarly early-stage SaaS companies.” His view, however, is that we need more data to discuss later-stage AI companies in the same terms: “[T]he data is still outstanding on whether mature AI businesses will be able to achieve the same 75%+ margins as SaaS businesses at scale.”
Why might things get better on the margin front as an AI company matures? In Kaufmann’s view, there is “room to optimize the machine learning toolchain, and this yet to-be-realized efficiency will hopefully lead to higher gross margins in the long-run.”
Finally, True Ventures’ Rohit Sharma made an interesting argument concerning hardware. Saying that AI-companies that have the sort of “50-60% gross margins” that a16z discussed are “typically companies with hardware-focused or hardware-related innovation that likely do not integrate software or intelligence in ways that are integral to products and more importantly, software that enables a unique capability for its users.” This leads to lower gross margins. However, AI companies with a “a clear drive toward using proprietary hardware in service of creating and deploying innovative software” have margins “closer” to what we see in SaaS.
Thematically, then, our group of investors agrees that today — and especially in early-stage AI-focused startups — gross margins are lower than what we might see in SaaS businesses. However, they are optimistic that things will get better with both tech improvements and falling compute costs.
Valuations
After asking about gross margins, we wanted to know what impact a lower band of results might have on AI startups’ valuations. So, we asked the investors that if they agreed on the margin point (they mostly did), did the different economics “impact how [they] have valued startups and other companies that leverage AI, when compared to SaaS companies and startups?”
Going in reverse order, Sharma told TechCrunch that while “SaaS frameworks are excellent operational frameworks for companies but are less useful in terms of offering insights for AI-related investments,” a rebuke that I accept; with AI-focused startups, gross margins are just one part of the puzzle. He added that “where there is at least the structural possibility of sustaining upward of 70% gross margins” that the market will see a “concentrated flight-to-quality effect in valuations.” The implication there is that AI companies that can’t meet that threshold might get left behind.
Kaufmann said long-term AI gross margins will improve, so he mostly disagreed with the premise of the question (lower long-term AI gross margins) and declined to answer.
Washburn had an optimistic note, saying that “[l]ower margins don’t overall impact valuations for AI business application software companies,” because “[u]ltimately, AI business application software has healthy recurring revenue and healthy margins that is delivering a unique and repeatable value proposition.”
Finally, on this question we return to Blume. Blume echoed an a16z point regarding the “commoditization of AI models and challenges,” saying that as “AI has become more democratised through open source projects like TensorFlow and PyTorch, the bar for this valuation premium becomes higher.” He’s bullish about long-term AI margins and valuations but says companies in the space “may take longer to reach” SaaS-like “ability to scale rapidly and predictably,” which could lead to a “different valuation profile over time for an AI company to a traditional SaaS startup, particularly over the first few rounds.”
Summarizing the above, I’d say the answer averages to “somewhat.” It appears that investors have a generally bullish view of AI companies (unsurprisingly), even if they note some caveats in how to value the firms. I’d say that SaaS still comes out on top here, but that AI valuations were given more credence than I anticipated, given the a16z post.
Optimism?
Our last question for the four investors was simple: “What is your optimism level today regarding AI startups and companies, and how has it changed over the last 12 months?”
Blume broke the AI world into two categories: Vertical AI, companies that tackle “specific problems which are well addressed by technologies,” and horizontal AI companies working to build “the infrastructure to help develop, deploy and manage this technology.” In his view, the latter category may struggle as market adoption of AI products might be slower than his firm had originally expected, as horizontal AI firms “rely on end market adoption to scale.”
Washburn had a slightly more rosy response, saying that he is still “extremely enthusiastic about the benefits of machine learning and deep learning to businesses,” saying that Intel Capital is actively in “AI-specific hardware, AI infrastructure software and AI business application software.” As a corporate venture capitalist, it was also no surprise that he wrapped his point with this: “In addition, given the compute intensity of AI applications, the demand for high performance and power efficient HW solutions will continue to grow, which drives a large opportunity for Intel.”
Play for the home team, and so forth.
Kaufmann sees a “growth crunch” in the SaaS market as “many cloud markets saturate.” (More on this point from my chat with another Scale Venture Partners partner here.) This will make AI-focused products and companies more attractive, as they won’t get caught up in a purely SaaS versus SaaS dust-up. Summarizing his own points, Kaufmann told TechCrunch that while “certain sub-fields in the discipline may be over-hyped like level 4 autonomy,” his firm is “broadly optimistic about the progress of AI, with the caveat that there is a ton of hype and irrational marketing promises in the discipline.”
Wrapping, let’s give Sharma the mic, who wrote that his “optimism level is markedly higher than it was a year ago for this area.” What’s making him feel bullish? Seeing “compelling founders with expertise and experience across machine intelligence applied in fundamentally new ways to very large verticals within just about every industry,” and seeing founders get into ethical questions early.
In good news, Sharma told TechCrunch that he’s seeing “significantly less AI-washing in startup pitch decks,” which must be welcome.
The full set of VC responses
With varying degrees of breadth, our VCs are pretty bullish on AI, caveats in mind. But don’t take our short summary as gospel, what follows are the full set of answers from the investors. Enjoy!
Ben Blume: Atomico
Does your experience investing in AI-focused and related companies and startups support the idea that these businesses have gross margins that are “in the 50-60% range” instead of the higher band often seen by SaaS companies?
It is true that the compute costs related to AI training and inference are substantial and this does impact an AI company’s gross margin, but these technologies enable companies to create great impact and solve hard and valuable problems. Lower than traditional SaaS gross margins can be seen as the “cost of doing business” to unlock the high ACV contracts that the most effective AI applications are able to command.
Compute costs have historically trended downwards though, and while this is plateauing, a whole wave of new AI chipsets like the Graphcore IPU are coming to market which should further drive this trend, reducing these costs substantially. There is also potential to optimise compute intensive products, and while we advise most startups not to focus on this early on while they are still engineering resource constrained and rapidly iterating, over time the best engineers can often squeeze out a multiple percentage point improvement in gross margin through algorithmic optimisation and clever use of compute resources.
The cost of the human in “human-in-the-loop” AI systems also has a clear impact on margin, and we see this in companies who rely heavily on data sets labelled by domain experts for training highly domain or customer specific models. We have seen many AI companies miss plans through underestimating the time it will take to reach sufficient levels of accuracy in a fully automated way, particularly as in many of these applications the edge cases are the most important ones. We see this as only a narrow subset of all AI companies and products though, and a lot of the time real value can come from applying simpler or narrower AI techniques, or models trained on pre-labeled or easy to label data as a component or feature of a larger product.
If so, does this impact how you have valued startups and other companies that leverage AI, when compared to SaaS companies and startups?
When we think about valuation we put a premium on companies with highly differentiated and defensible technology. As AI has become more democratised through open source projects like TensorFlow and PyTorch, the bar for this valuation premium becomes higher, and the differentiation is increasingly not in the core AI but in the way that this is applied to a product, or combined with other algorithmic or analytical techniques which could be more differentiated.
SaaS companies in general command premium multiples due to their ability to scale rapidly and predictably once their model is working. In the end this should hold true for AI companies at scale too, though it may take longer to reach this state due to the increased complexity of building an effective product, finding product market fit, and establishing the best sales and adoption journey.
Overall this means a different valuation profile over time for an AI company to a traditional SaaS startup, particularly over the first few rounds, and as investors we need to take this into account in our models and forecasts.
What is your optimism level today regarding AI startups and companies, and how has it changed over the last 12 months?
We generally don’t think about AI startups as collective in this way, rather we look at “vertical AI” businesses in industry verticals or functional areas that have specific problems which are well addressed by technologies such as classification, anomaly detection or machine vision, and “horizontal AI” businesses which form the infrastructure to help develop, deploy and manage this technology.
As we speak to the potential buyers of these technologies, we are increasingly optimistic in the long term opportunity that exists to apply AI to a wide range of problems across industries and sectors. However we are more cautious than 12 months ago about the pace at which these buyers will adopt these products, which will likely be slower than many had anticipated, particularly in industries which have seen less technology disruption to date.
This in turn impacts the growth of horizontal AI infrastructure companies, which rely on end market adoption to scale. However as AI eventually permeates deeper into organizations, the opportunity for new and effective tooling to enable this will be a fast growing and evolving one.
Nick Washburn: Intel Capital Partners
1. I agree with the early observation that some AI business application software companies run at overall lower gross margins than some traditional non-AI SaaS companies. There are several key drivers of this observation:
- Both traditional non-AI SaaS companies and AI business application companies carry cloud infrastructure costs, but the computational costs of model development and deployment on the AI-side drives up COGS. AI business application software is naturally more data intensive, so the overall cloud-bill for storage and data access often runs higher. This compute and data intensity for both experimentation/training and deployment/inference carry a higher cloud cost. We have seen AI companies drive cutting-edge computational strategies (i.e., running NLP models in a “serverless” fashion using Lambda functions), as AI infrastructure engineers try to cut cost and increase performance. Similar approaches on distributed computing, but that drives more speed and scale than cost.
- Human-in-the-loop is also a critical distinction. When there is human-in-the-loop inputs of subject matter experts on the training/experimentation side, this will include a services component. There are often more complex integrations into customer environments depending on the use cases identified. As AI business applications have moved from more horizontal to more vertically focused, some of these levers have lessened, but they are still present.
- It is important to distinguish between “AI services companies” and “AI product companies with a services component”, though. A lot of this distinction is driven by the customer’s use cases and how much is bespoke/customized. There are some exceptions, but more vertically-focused applications tend to fall more into the AI product with services component category, than more horizontal business software.
2. Over time, AI business application software and non-AI SaaS companies will converge particularly in categories with a specific vertical focus. Rules-based systems, more complex machine learning and deep learning continue to bleed into traditional SaaS software and will evolve. While that does bring some margin pressure, as business customers become more focused on the benefits and efficiencies driven by some version of AI, these worlds will converge. Cloud infrastructure software continues to be evenly split, as the tools for the traditional software development lifecycle do not work for machine learning at scale (and there is a big opportunity there for AI infrastructure companies).
3. Lower margins don’t overall impact valuations for AI business application software companies. Healthy margins are still achievable, but these drivers of margin pressure need to be understood and built into the business model. Martin and Matt did a great job characterizing the issue of “hidden R&D costs” and how to manage appropriately. Ultimately, AI business application software that has healthy recurring revenue and healthy margins that is delivering a unique and repeatable value proposition to line of business buyers is still valued very high and poses a great opportunity.
4. I remain extremely enthusiastic about the benefits of machine learning and deep learning to businesses. Intel Capital continues to invest heavily in the space, both for AI-specific hardware, AI infrastructure software and AI business application software. We have a deep understanding of these businesses given we’ve been investing in all layers of the AI stack for years, and also feel we have a unique value proposition to founders to really add value, whether technical or go to market. It is an exciting time to be investing in enterprise software. In addition, given the compute intensity of AI applications, the demand for high performance and power efficient HW solutions will continue to grow, which drives a large opportunity for Intel.
Jeremy Kaufmann: ScaleVP
Does your experience investing in AI-focused and related companies and startups support the idea that these businesses have gross margins that are “in the 50-60% range” instead of the higher band often seen by SaaS companies?
I believe that the initial gross margin for many AI companies will absolutely be lower than we see for similarly early-stage SaaS companies, but the data is still outstanding on whether mature AI businesses will be able to achieve the same 75%+ margins as SaaS businesses at scale. For reference, our Scale index of 68 public SaaS companies shows a median gross margin of 73% for public SaaS businesses.
In the early stages, an AI company is likely doing custom work to solve the AI “cold-start problem” which reduces early gross margins. The cold-start problem is that AI companies need a way to acquire the relevant data before they can provide the requisite value to customers and ultimately convert them to paying customers. Therefore in the initial implementation, an AI business might start with early models that are company specific to gather this initial data, and then over time as they can pool data across customers, will then become use case and/or industry specific and realize higher gross margins. So I agree that initial margins will be lower.
However, looking over the long-run, it is important to note that we are still in the early days of managing the deployment, training and compute of AI models so there is lots of room for improvement here, which will hopefully allow AI companies to potentially realize higher gross margins in the future. Just as the sophistication of the modern-day software deployment process was helped along by innovations such as continuous integration and Git, there is corresponding room to optimize the machine learning toolchain, and this yet to-be-realized efficiency, will hopefully lead to higher gross margins in the long-run.
While the recent trends are certainly towards higher complexity models with more parameters using more compute resources in fields like language processing, it is important to note that in other sub-fields like computer vision, we’ve seen vast improvements in training time. For example, the Stanford AI Index Report of 2019 shows that using the Stanford DAWN project as a proxy for the ImageNet dataset, over the last year and a half, the time required to train a network for supervised image recognition has fallen from three hours in October 2017 to 88 seconds in July 2019. The training cost as measured by the cost of public cloud instances to train this corresponding image classification model fell from $2,323 to $12 between October 2017 and October 2018. I hope this trend is suggestive of lower compute costs after a given AI sub-discipline has had time to mature, with the caveat that newer fields like video processing and language understanding are still rapidly undergoing change.
If so, does this impact how you have valued startups and other companies that leverage AI, when compared to SaaS companies and startups?
N/A
What is your optimism level today regarding AI startups and companies, and how has it changed over the last 12 months?
We are very optimistic about AI startups and the core theme of our fund is investing in the “intelligent connected world.”
Broadly speaking, we believe the last 2 decades of enterprise software since Salesforce in 1999 was the replacement of on-premises software with cloud software, but within the next five years, there will be a “growth crunch” as many cloud markets saturate. At that point, the Cloud Gold Rush will become the Hunger Games as cloud companies large and small compete against each other for survival.
There will be three winning strategies for a startup when this happens: Fight, or compete head on in an existing cloud market with a better UI; Focus, or find those parts of the cloud market where there is still low competition and good growth such as in the vertical markets; or Fly, which is to build a company based on more than just the move to the cloud, and we believe AI is essential to this third strategy.
In particular, building “beyond the cloud” means “assume the cloud” and building on top of that stack using newer technologies and a design approach where instead of the user working for the software, the software works for (or instead of) the user. At Scale we think of this as building the Intelligent Connected World.
Over the last decade, advances in computer vision have driven a great deal of the progress in AI, from Google’s famous ImageNet dataset to newer applications like being able to spot irregularities in medical scans. However, if we narrow our focus to just the last year versus the last decade, recent progress in NLP has been staggering. Transfer learning from pre-trained language models is ushering in an “ImageNet moment” for NLP.
Historically, most datasets for supervised NLP tasks have been rather small, which makes it difficult to train deep neural networks, as they tend to overfit on these small training datasets and don’t generalize well in practice. In comparison, for the last few years in computer vision, the trend has been to pre-train any model on the ImageNet dataset, which has been a significant advantage and explains why computer vision has shown a faster rate improvement against the standard benchmarks compared to NLP.
Recent breakthroughs like Google’s BERT and ELMo can be considered the ImageNet equivalent for NLP. Among the biggest benefits of pre-trained language modeling is the fact that training data comes for free with any text corpus and that potentially unlimited amounts of training data is now available.
With this trend towards pre-training language models, progress as measured by traditional NLP benchmarks like GLUE has been greater in the past year than any previous year to date. The state-of-the-art has increased from a score of 69 to 88 over 13 months, where the human baseline is 87. To give context as to how large of an improvement this is, researchers were actually forced to introduce a new benchmark (SUPERglue)!
While certain sub fields in the discipline may be over-hyped like level 4 autonomy, we continue to be broadly optimistic about the progress of AI, with the caveat that there is a ton of hype and irrational marketing promises in the discipline.
Rohit Sharma: True Ventures
Does your experience investing in AI-focused and related companies and startups support the idea that these businesses have gross margins that are “in the 50-60% range” instead of the higher band often seen by SaaS companies?
AI-focused and AI-related companies with 50-60% gross margins are typically companies with hardware-focused or hardware-related innovation that likely do not integrate software or intelligence in ways that are integral to products and more importantly, software that enables a unique capability for its users. Where there is a clear drive toward using proprietary hardware in service of creating and deploying innovative software, we see margins closer to those of pure-software companies.
Historically, some of the largest venture returns have been delivered in the area of communications and in the data-infrastructure era of companies like Cisco and Juniper, which delivered innovative software and services atop custom hardware to massively enhance workplace productivity. The most innovative companies in this domain maintained 60%+ gross margins and even higher for specialty solutions like security or data storage appliances etc.
I see AI and robotics startups through a similar lens. Most companies in this area will create new and useful machines, but the most impactful will be the ones that deliver software-driven features and services that outlast multiple generations of hardware underneath. Those that have temporary (hardware) innovations will regress to ~50% or lower gross margins, while startups that impact business value and behaviors will rise to more than 80% gross margins. For example, server manufacturers in absence of unique OS/user-level features regressed to commodity-hardware margin structures in the 20-30% range, while the makers of security software with substantially similar complexity in terms of hardware remained at significantly higher margins. It was interesting to note that wifi access point and switching solutions makers had even higher gross margins than data-networking systems.
I don’t quite agree that these businesses somehow carry intrinsically disproportionately higher compute costs. Data-centric computing solutions alongside heterogenous computing solutions (x86, GPUs, ASICs, inference engines…) are poised to provide dramatically higher performance at the same or lower cost in the next few years. I think there is a massive uplift in semiconductors and switching systems underway that keeps infrastructure (computing/cloud) cost equations at approximately the same factor as it was for large scale enterprise or consumer software efforts in the cloud-era. I think the larger factor is the fear that these solutions may not provide immediately valuable insights and economic value to the customers and therefore may see compressed margins till they do. Margin structures in AI will not be driven by the cost structure of the solutions, rather they will be determined by the degree of economic impact delivered to the customers at scale.
If so, does this impact how you have valued startups and other companies that leverage AI, when compared to SaaS companies and startups?
I believe SaaS frameworks are excellent operational frameworks for companies but are less useful in terms of offering insights for AI-related investments. Valuations will be a function of the quality of the eventual outcome, and a high (Gross and Net) margin is one aspect of that. In cases where there is at least the structural possibility of sustaining upward of 70% gross margins, as we saw with generations of data networking and security specialist hardware companies, we will see a concentrated flight-to-quality effect in valuations.
What is your optimism level today regarding AI startups and companies, and how has it changed over the last 12 months?
My optimism level is markedly higher than it was a year ago for this area. We are seeing more compelling founders with expertise and experience across machine intelligence applied in fundamentally new ways to very large verticals within just about every industry. This goes beyond just enterprise office work, marketing, or sales, etc. We are also seeing significantly less AI-washing in startup pitch decks, which probably peaked 9-12 months ago. And something that is very satisfying for us at True is that we are also seeing founders discuss the ethics of AI-related data at the company-formation stage, not as an afterthought like the industry saw with privacy-related data. Broadly speaking, “AI” is not a market that exists yet, so we assign the sum of our fears and expectations to this still nascent and emerging market.