Inference-side flywheel
Reports
Notes
- Intelligence flywheel + Cost flywheel + User stickiness
- If true, usage with high quality feedback will be a key moat, and frontier models will stay on the frontier.
- Intelligence flywheel: more usage -> more feedbacks (intelligence signals) -> better models and products -> more usage.
- Mechanisms:
- RL from human feedbacks (RLHF) / failures / outcomes (e.g. coding),
- more evaluations,
- domain knowledge,
- tool-use / workflow,
- synthetic data generation.
- Feedback quality is key:
- expert feedback, automated verification
- task success / failure
- user corrections
- likes/dislikes
- Mechanisms:
- Cost flywheel: more usage -> lower cost per task through hardware scale (lower compute), better software (higher efficiency), better models (less compute needed) -> more usage.
- Mechanisms:
- better scale and utilization,
- better scheduling and batching,
- optimization data (e.g. shorter reasoning, better prompts, smaller models),
- model specialization and routing,
- distillation,
- quantization,
- hardware optimization,
- hardware learning curve (GPUs,
- inference accelerators, networking chips, memory),
- custom inference chips (e.g. Google TPU, ASW Inferentia, custom silicon),
- software ecosystem improvements (inference frameworks, kernes, compilers, libraries),
- agentic workflow optimization.
- Mechanisms:
- Examples & implications cf
- Data and model lock-in / User stickiness, for both enterprises and consumers.
- Meta ad gen model usage flywheel
- Grok models benefit from the coding usage in Cursor.
- OpenAI is harmed by the consumer adoption of ChatGPT with no particularly useful data for feedback.
- Frontier labs' fight with enterprises for data, IP and customer ownership.
Prompt
Research on inference-side flywheel, both performance flywheel and cost flywheel, the underlying mechanisms, implications for frontier labs, AI compute economics and AI competitive landscape.
