Inference Is the Most Important Market in Software

Overview

Inference is becoming the core of software applications, marking a significant structural change in the industry as vendors adapt to shifting market dynamics. This evolution is leading to increased costs for software vendors and consumers alike, as gross margins may decline and infrastructure expenses rise, while pricing trends show a substantial drop in capability-adjusted token costs, juxtaposed with a sharp increase in the frontier blended price.

Key points

Market Dynamics of Software Inference

• Inference is becoming the central market in software.
• Every application is evolving to sell inference.
• This represents a significant structural change in the software industry.

Cost Implications for Software Vendors

• Inferences will lead to higher consumption costs than base subscription fees, altering compensation and complicating customer forecasting.
• Gross margins are expected to decline below standard SaaS levels unless applications create proprietary methods to reduce costs effectively.
• Infrastructure expenses now represent a major portion of total costs in the application layer.

Pricing Trends

• Capability-adjusted token costs decline significantly, dropping around 47% each quarter.
• The frontier blended price increased dramatically from $0.96 to $11.25 in 18 months.