ai-cost-audit
Audit AI usage, billing, unit economics, routing, caching, batching, and optimization. Use when AI cost, margin, pricing, model choice, or savings must be measured or verified.
- Category
- ai
- Package
- ai-cost-audit/SKILL.md
- License
- MIT
- Author
- @tushaarmehtaa
- Tags
- aimodelscostroutingmarginscachingbatch
Install
Swipe for more runtimes.
Codex
Skills directory: ~/.codex/skills
Install globally
npx skills add tushaarmehtaa/tushar-skills --skill ai-cost-audit -g -a codex -yInvoke
$ai-cost-audit or /skillsYou can also describe the task naturally; runtimes may select the skill from its description.
Required access
Claude Code
Skills directory: ~/.claude/skills
Install globally
npx skills add tushaarmehtaa/tushar-skills --skill ai-cost-audit -g -a claude-code -yInvoke
/ai-cost-auditYou can also describe the task naturally; runtimes may select the skill from its description.
Required access
Cursor
Skills directory: ~/.cursor/skills
Install globally
npx skills add tushaarmehtaa/tushar-skills --skill ai-cost-audit -g -a cursor -yInvoke
/ai-cost-auditYou can also describe the task naturally; runtimes may select the skill from its description.
Required access
Claude app
This workflow can run in chat using the files and context you provide. Download its complete ZIP, then upload it from Claude's Skills settings.
ChatGPT Skills
This workflow is suitable for ChatGPT Skills. ChatGPT does not document the same upload archive format as Claude, so follow its uploader instead of reusing the Claude ZIP.
ChatGPT upload guide →Instructions
Source: SKILL.mdAI cost audit
Build a cost model from observed usage and current primary-source prices. Treat repository intent, measured usage, invoices, and projections as different evidence classes.
Choose a mode
- Inventory: map model/media calls, routing, retries, and ownership.
- Economics: calculate cost per action, user, plan, and month.
- Simulation: compare pricing, volume, model, cache, batch, or abuse scenarios.
- Optimization: rank changes after measuring quality and operational risk.
- Reconciliation: explain the gap between bottom-up estimates and provider invoices.
- Verification: confirm that a completed change reduced spend without unacceptable quality or latency regressions.
Use the narrowest mode that answers the request. Combine modes only when the user asks for a full audit or the dependency is necessary.
Workflow
- Inspect the repository, existing telemetry, billing exports, pricing configuration, and prior analyses before asking questions.
- State the audit boundary: environments, date range, providers, features, currencies, taxes, credits, and whether non-model infrastructure is included.
- Inventory direct and indirect calls: generation, reasoning, embeddings, reranking, tools, image/audio/video, moderation, retries, fallbacks, agents, queues, evaluations, and batches.
- Prefer provider-metered tokens or media units. Keep measured values, code-derived estimates, generic estimates, and assumptions visibly separate. Use p50, p95, and worst-case where available.
- Fetch current prices only from official provider sources when pricing affects the answer. Record URL, retrieval date, region/tier/currency, and special terms such as cached input, reasoning tokens, batch, storage, or minimum charges. Do not rely on bundled price tables or memory.
- Model each cost path, including failed calls, retry amplification, tool loops, cache writes/reads, storage, egress, gateway fees, payment fees, free allowances, and shared fixed costs when relevant.
- Reconcile the modeled total against invoices or billing dashboards. Quantify unexplained variance instead of forcing agreement.
- Run normal, high-usage, abuse, and sensitivity scenarios. Do not apply universal margin or traffic thresholds without the product's business constraints.
- Rank recommendations by expected savings range, evidence confidence, quality risk, latency effect, engineering effort, reversibility, and measurement plan.
- Require an evaluation and canary before changing models, prompts, routing, or output limits. Verify spend, quality, latency, error rate, and user outcomes afterward.
Ask only for inputs that cannot be recovered from the scoped artifacts and materially change the result.
Safety and evidence rules
- Never invent usage, conversion, revenue, invoice, cache-hit, or quality inputs.
- Never expose secrets found in environment files or billing exports.
- Do not recommend a cheaper model solely from task labels; require representative evaluation data.
- Label projections as projections and show formulas with units.
- Treat prices and model availability as time-sensitive.
Load conditional references
- Read model inventory for discovery patterns, call-path fields, and routing analysis.
- Read unit economics for formulas, reconciliation, scenario design, and recommendation ranking.
These references provide specialist detail only. This file controls mode, interaction, safety, output, and verification.
Output contract
Return only sections supported by the selected mode:
- scope and evidence ledger;
- model/media call inventory with file paths and ownership;
- assumptions and data gaps;
- formulas with units and a scenario table;
- highest-cost paths and invoice reconciliation;
- prioritized changes with savings ranges and quality gates;
- verification results or an explicit measurement plan.
Cite every time-sensitive price with an official source and retrieval date.
Verify
- Inventory covers wrappers, retries, fallbacks, background jobs, and non-text operations.
- Totals preserve units, currencies, date ranges, and environment boundaries.
- Measured and estimated inputs are distinguishable.
- Bottom-up totals are reconciled to billed totals when billing is available.
- Recommendations include a quality gate, rollback path, and owner or next action.
- Completed optimizations are verified with before/after spend, quality, latency, and error data.
Bundled references
2 files · 213 lines
references/model-inventory.md
source ↗Model and media call inventory
Use this reference only after the main skill has selected an inventory, economics, reconciliation, optimization, or verification mode. Do not restart intake here.
Discovery strategy
Search with rg, then follow indirection rather than trusting model literals alone.
Look for:
- provider SDK constructors and request methods;
- gateways, proxy clients, model aliases, environment variables, and configuration files;
- orchestration frameworks, agent/tool loops, evaluators, and structured-output helpers;
- embeddings, rerankers, moderation, OCR, transcription, speech, and image/video operations;
- queues, cron jobs, workers, webhooks, migrations, notebooks, tests, and admin tools;
- retry, fallback, timeout, circuit-breaker, and provider-routing code;
- usage logging, billing webhooks, cost tags, budgets, and tracing spans.
Start broad, then inspect every returned call path. Provider method names change; do not rely on a frozen pattern list.
Inventory fields
Record one row per distinct priced path, not merely per source line:
| Field | Meaning |
|---|---|
| Feature/action | User or system outcome that triggers the path |
| Location | File, function, worker, or external workflow |
| Environment | Production, staging, local, evaluation, migration |
| Provider/model | Resolved runtime value and alias source |
| Operation | Generation, reasoning, embedding, media, tool, rerank, moderation |
| Input source | Prompt, history, retrieval, files, media, tool results |
| Output control | Token/media limit, schema, stop rule |
| Frequency | Per request/action/session/job and observed volume |
| Routing | Eligibility, difficulty tier, provider/model selection |
| Amplification | Retries, fallbacks, loops, fan-out, regeneration |
| Cache/batch | Eligibility, configuration, observed hit/use rate |
| Latency | User-facing requirement and observed distribution |
| Usage evidence | Provider meter, trace, log, code estimate, or assumption |
| Owner | Team or component responsible |
Resolve aliases to actual runtime values by environment. Preserve uncertainty when configuration is injected externally.
Call-graph checks
For each feature, trace:
- entrypoint and eligibility;
- preprocessing and context construction;
- primary request;
- tool calls or recursive turns;
- validation and repair requests;
- retry and fallback behavior;
- asynchronous follow-up work;
- stored artifacts and downstream media/embedding jobs.
Count amplification from code and telemetry separately. A nominal “one model call” feature may create several billed operations.
Optimization candidates
Flag candidates only with supporting evidence:
- Remove: call does not affect a user or operating decision.
- Reduce: avoidable context, duplicate retrieval, excessive output, or repeated repair.
- Cache: stable repeated prefix or result with compatible provider semantics and privacy boundary.
- Batch: asynchronous work whose latency requirement matches the provider's current batch terms.
- Route: measurable task classes differ in required quality or latency.
- Consolidate: scattered literals or equivalent calls prevent governance.
- Observe: usage cannot be measured or tied to a feature.
Do not infer a safe model downgrade from labels such as “classification” or “summary.” Build a representative evaluation set and compare quality, latency, and failure cost.
Inventory quality checks
- Search results include indirect wrappers and non-text operations.
- Runtime configuration is resolved for every relevant environment.
- Retries, fallbacks, tools, and fan-out are represented.
- Frequency and usage identify evidence class and date range.
- Unused configuration is distinguished from unreachable or externally triggered code.
- Secrets and raw customer content are excluded from the report.
references/unit-economics.md
source ↗Unit economics and reconciliation
Use these formulas after the main skill establishes scope and evidence. Fetch all prices from current official provider sources; this reference intentionally contains no model prices or universal margin benchmarks.
Contents
- Evidence hierarchy
- Per-operation cost
- Product economics
- Cache analysis
- Batch and routing analysis
- Reconciliation
- Scenario design
- Recommendation record
Evidence hierarchy
Prefer, in order:
- provider invoice/billing export;
- provider-metered usage attached to requests;
- application traces with token/media counts;
- tokenizer or file-duration/dimension calculations;
- code-derived limits and observed volume;
- explicitly labeled generic assumptions.
Never blend classes without showing which inputs are estimated.
Per-operation cost
For a token-priced operation:
input_cost = uncached_input_units × input_rate
cache_read_cost = cached_input_units × cache_read_rate
cache_write_cost = cache_write_units × cache_write_rate
output_cost = output_units × output_rate
reasoning_cost = billed_reasoning_units × reasoning_rate
request_cost = sum(components) + tool/media/minimum fees
Normalize rates and units before arithmetic. For media, use the provider's billed duration, resolution, characters, images, or other current unit.
Expected feature cost:
base_paths = Σ(path_probability × path_cost)
retry_cost = Σ(retry_probability × retry_path_cost)
fallback_cost = Σ(fallback_probability × fallback_path_cost)
feature_cost = base_paths + retry_cost + fallback_cost + storage + egress + gateway
Report a distribution where request shapes vary materially.
Product economics
variable_cost_per_action = AI + non-AI variable infrastructure + payment-variable cost
contribution_per_action = allocated_revenue - variable_cost_per_action
contribution_margin = contribution_per_action / allocated_revenue
monthly_variable_cost = Σ(action_volume × expected_action_cost)
gross_profit = recognized_revenue - cost_of_revenue
break_even_usage = available_contribution / marginal_action_cost
State the accounting boundary. Gross margin, contribution margin, and cash spend answer different questions.
For free allowances:
expected_free_cost = signup_count × activation_rate × E[cost | activated free user]
payback_ratio = contribution_per_payer / expected_cost_per_nonpayer
Do not invent activation, conversion, or usage distributions.
Cache analysis
Model provider-specific write/read rules, TTLs, minimum cacheable length, invalidation, and privacy boundaries from current documentation.
uncached_cost = repeated_units × normal_input_rate
cached_cost = write_cost + expected_reads × read_cost + uncached_remainder
cache_savings = comparable_uncached_cost - cached_cost
break_even_reads = incremental_write_cost / savings_per_read
Use observed prefix stability and hit rate. Include latency and quality effects when context must be restructured.
Batch and routing analysis
For batch, verify current eligibility, completion window, cancellation behavior, quota, and discount. Compare with the workload's real deadline and retry policy.
For routing, calculate expected savings using observed task mix, then gate the proposal with representative evaluations. Include false-success cost, retry amplification, latency, and operational complexity.
Reconciliation
Reconcile by provider, account/project, model/operation, environment, and day where possible.
variance = billed_total - modeled_total
variance_pct = variance / billed_total
Investigate taxes, credits, tiers, minimums, storage, fine-tuning, deleted logs, untagged environments, retries, external tools, currency conversion, and billing-period boundaries. Leave a residual unexplained amount if evidence cannot resolve it.
Scenario design
Include only decision-relevant scenarios:
- measured baseline;
- high-usage or tail request shape;
- abuse/retry incident;
- price or volume sensitivity;
- proposed cache/batch/routing change;
- quality failure or fallback amplification.
Show formulas, input values, provenance, and a range rather than false precision.
Recommendation record
For each change, report:
- affected path and evidence;
- monthly savings range and confidence;
- engineering effort and owner;
- quality, latency, privacy, and reliability risk;
- evaluation/canary design;
- rollback trigger;
- post-change measurement window.