The Continuity Layer
← The ScorecardCoding agents →Memory systems →Open models →Hardware, hosting, routers →
The Scorecard · Inference APIs

Inference APIs: what they charge, what they let you send, and what they keep

thecontinuitylayer.com/inference-apis

Together, Fireworks, Groq, OpenRouter, DeepInfra, Baseten and HiAPI, plus the makers' own APIs from OpenAI, Anthropic, Google, Mistral, DeepSeek, xAI and Cohere, read from their own pages and graded on the same five questions: a public per-token price, published rate limits, whether your prompts train their models, how long they keep what you send, and a public status page.

14
Inference APIs graded
5
Tests
70
Sourced answers
Docs
Basis
Read from the makers' own pages between 3 and 5 November 2026

All fourteen publish a price, twelve of them a plain per-token price with no sign-in, and Groq, OpenAI and Anthropic answer all five questions in your favor. The split is on training: six say plainly that API prompts are not used for training by default, five make it depend on the plan, the tier or the upstream provider, and Baseten, DeepSeek and HiAPI say nothing on the pages that would open; every grade is read from the provider's own page on the day shown, and no speed benchmark appears here.

At a glance

Every product, every test.

✓ yes, ½ partly, ✕ no, – the maker's pages do not say. A dash counts for nothing: a product gets credit only for what its own pages state, so a vague page never outranks a clear one. On the training test the colors flip, because no is the good answer there.

Sort by
Inference APIs: each product's grade on each test, and its score out of all the tests
ProductPriceRate limitsTrainingRetentionStatus pageScore
Groq#1 · GroqA+5/5
OpenAI API#2 · OpenAIA+5/5
Anthropic API#3 · AnthropicA+5/5
Fireworks AI#4 · Fireworks AIA4½/5
OpenRouter#5 · OpenRouterA4½/5
DeepInfra#6 · DeepInfraA4½/5
Gemini API#7 · GoogleB+4/5
xAI API#8 · xAIB+4/5
Together AI#9 · Together AIB3½/5
Mistral API#10 · Mistral AIB3½/5
Cohere API#11 · CohereB3½/5
Baseten#12 · BasetenB3½/5
DeepSeek API#13 · DeepSeekC−2½/5
HiAPI#14 · HiAPID2/5
Product by product

In score order, best first.

  • Groq

    #1 · Groq

    An inference API that serves open models on Groq's own hardware, billed per token.

    A+5/5On the 5 tests
    • Does Groq publish a per-token price?Yes
    • Does Groq publish its rate limits?Yes
    • Does Groq train on your API prompts by default?No
    All 5 answers →

  • OpenAI API

    #2 · OpenAI

    OpenAI's own API for its GPT models.

    A+5/5On the 5 tests
    • Does OpenAI API publish a per-token price?Yes
    • Does OpenAI API publish its rate limits?Yes
    • Does OpenAI API train on your API prompts by default?No
    All 5 answers →

  • Anthropic API

    #3 · Anthropic

    Anthropic's own API for its Claude models.

    A+5/5On the 5 tests
    • Does Anthropic API publish a per-token price?Yes
    • Does Anthropic API publish its rate limits?Yes
    • Does Anthropic API train on your API prompts by default?No
    All 5 answers →

  • Fireworks AI

    #4 · Fireworks AI

    A hosted inference platform for open models, with serverless per-token pricing and dedicated deployments.

    A4½/5On the 5 tests
    • Does Fireworks AI publish a per-token price?Yes
    • Does Fireworks AI publish its rate limits?Partly
    • Does Fireworks AI train on your API prompts by default?No
    All 5 answers →

  • OpenRouter

    #5 · OpenRouter

    One API key that routes requests to many model providers, with the per-token price set by the provider used.

    A4½/5On the 5 tests
    • Does OpenRouter publish a per-token price?Yes
    • Does OpenRouter publish its rate limits?Yes
    • Does OpenRouter train on your API prompts by default?Partly
    All 5 answers →

  • DeepInfra

    #6 · DeepInfra

    A hosted inference platform for open models, billed per token.

    A4½/5On the 5 tests
    • Does DeepInfra publish a per-token price?Yes
    • Does DeepInfra publish its rate limits?Yes
    • Does DeepInfra train on your API prompts by default?Partly
    All 5 answers →

  • Gemini API

    #7 · Google

    Google's own API for its Gemini models, served through AI Studio.

    B+4/5On the 5 tests
    • Does Gemini API publish a per-token price?Yes
    • Does Gemini API publish its rate limits?Partly
    • Does Gemini API train on your API prompts by default?Partly
    All 5 answers →

  • xAI API

    #8 · xAI

    xAI's own API for its Grok models.

    B+4/5On the 5 tests
    • Does xAI API publish a per-token price?Yes
    • Does xAI API publish its rate limits?Yes
    • Does xAI API train on your API prompts by default?No
    All 5 answers →

  • Together AI

    #9 · Together AI

    A hosted inference platform that serves open models through one API, billed per token.

    B3½/5On the 5 tests
    • Does Together AI publish a per-token price?Yes
    • Does Together AI publish its rate limits?No
    • Does Together AI train on your API prompts by default?No
    All 5 answers →

  • Mistral API

    #10 · Mistral AI

    Mistral AI's own API platform, La Plateforme, for its models.

    B3½/5On the 5 tests
    • Does Mistral API publish a per-token price?Yes
    • Does Mistral API publish its rate limits?No
    • Does Mistral API train on your API prompts by default?Partly
    All 5 answers →

  • Cohere API

    #11 · Cohere

    Cohere's own API for its Command models.

    B3½/5On the 5 tests
    • Does Cohere API publish a per-token price?Partly
    • Does Cohere API publish its rate limits?Yes
    • Does Cohere API train on your API prompts by default?Partly
    All 5 answers →

  • Baseten

    #12 · Baseten

    An inference platform for running models in production, with a per-token Model API and dedicated deployments.

    B3½/5On the 5 tests
    • Does Baseten publish a per-token price?Partly
    • Does Baseten publish its rate limits?Yes
    • Does Baseten train on your API prompts by default?Not stated
    All 5 answers →

  • DeepSeek API

    #13 · DeepSeek

    DeepSeek's own API for its models.

    C−2½/5On the 5 tests
    • Does DeepSeek API publish a per-token price?Yes
    • Does DeepSeek API publish its rate limits?Partly
    • Does DeepSeek API train on your API prompts by default?Not stated
    All 5 answers →

  • HiAPI

    #14 · HiAPI

    One API key for text, image, video and audio models from many providers, with text models billed per token and media models per image, per second or per request.

    D2/5On the 5 tests
    • Does HiAPI publish a per-token price?Yes
    • Does HiAPI publish its rate limits?Partly
    • Does HiAPI train on your API prompts by default?Not stated
    All 5 answers →
The tests

5 questions, the same for every inference API.

  1. 1

    Does a inference API publish a per-token price?

    Yes means: Per-token prices are on a public page with no sign-in. Partly means only some models are priced in public, or the price sits on a model page rather than the pricing page.

  2. 2

    Does a inference API publish its rate limits?

    Yes means: The docs give request or token numbers for each tier or model. Partly means some numbers, such as a concurrency cap or the top tiers only; no means the numbers are only shown inside a signed-in account.

  3. 3

    Does a inference API train on your API prompts by default?

    Yes means: Here no is the good answer: the provider's page says what you send its API is not used for training unless you opt in. Yes means it is unless you opt out; partly means it depends on the plan, the tier or the upstream provider.

  4. 4

    Does a inference API say how long it keeps your prompts and outputs?

    Yes means: A stated period, or a statement that nothing is kept by default. Partly means a period for some users or requests but not others.

  5. 5

    Does a inference API run a public status page?

    Yes means: A public page shows current uptime and past incidents.

Every grade is read from the maker's own documentation or pricing page on the day shown, with the page linked on the product's page. Nothing is graded from a review, a blog or a test we ran; where we test a product ourselves the page will say so. If a page has changed, tell us.

Questions people ask

The short answers.

Which inference APIs say they do not train on my prompts?
Together AI, Fireworks AI, Groq, OpenAI, Anthropic and xAI say API prompts are not used for training unless you opt in. OpenRouter, DeepInfra, Google's Gemini API, Mistral and Cohere make it depend on the upstream provider, the tier or an opt-out, and the Baseten, DeepSeek and HiAPI pages that would open say nothing. Each provider's page gives the line in plain words and links to it.
Which inference APIs publish their rate limits?
Groq, OpenRouter, DeepInfra, Baseten, OpenAI, Anthropic, xAI and Cohere publish numbers in their docs. Fireworks, Google, DeepSeek and HiAPI publish some of them, and Together AI and Mistral give none in their public docs.
How long do inference APIs keep my prompts?
OpenAI, Anthropic, Groq, Mistral and xAI state up to about 30 days of logs for abuse checks, Google states 55 days for the Gemini API, and Fireworks, DeepInfra, OpenRouter and Baseten say standard requests are not stored by default. HiAPI states 90 days for API call logs and about 7 days for temporary outputs but no period for prompts. Together AI stores prompts by default with no stated limit, Cohere gives no period for trial users, and DeepSeek's API privacy page would not open.

Placement here is never for sale and no link on this page pays us. How links work on this site. The consumer assistants are graded the same way on the Scorecard.