Managing AI spend inside a FinOps practice

AI did not replace cloud budgets — it stacked on top of them. GPU training, inference endpoints, vector databases, and third-party model APIs create new cost shapes that classic VM FinOps only partially covers.

Inform first: make AI a first-class cost dimension

• Separate training vs. inference in tags and accounts/projects so spikes are diagnosable.
• Include third-party API spend (OpenAI, Anthropic, etc.) in the same ledger as hyperscaler GPU — otherwise “AI cost” is incomplete.
• Define unit metrics early: cost per 1k tokens, cost per prediction, cost per experiment — not only total monthly spend.

Where waste usually hides

• Idle GPU nodes left up between experiments.
• Over-sized inference fleets for traffic that never arrived.
• Duplicate datasets and unchecked storage growth next to model artifacts.
• Unbounded prompt/context sizes that inflate token bills without improving product quality.

Operate with the same FinOps loop

Apply Inform → Optimize → Operate to AI the same way you do to EC2: budgets and anomaly alerts per AI product line, weekly reviews with owners, and commitment or capacity planning only after the baseline is stable. The State of FinOps data shows most organizations are still building visibility — do not skip to exotic optimization before allocation works.