# FasterTokens — full documentation

> Large models like DeepSeek V4.1 Flash are smart enough for a phone call, but on any one provider their first token keeps stalling for more than a second or timing out. FasterTokens is an OpenAI-compatible endpoint that queries multiple inference providers simultaneously on every turn and optimizes for time to first token. Latency is measured around the clock and published.

Source: https://fastertokens.com

# Quickstart

FasterTokens is an OpenAI-compatible chat completions endpoint for voice agents. For every request it queries multiple inference providers simultaneously and optimizes for time to first token. You change a base URL and a key; your prompts, tools and streaming code stay as they are.

## 1. Create an API key

[Sign up](https://fastertokens.com/signup), open **API keys** in the dashboard and create a key. It starts with `ft_live_` and is shown once. Keep it on your server:

```bash
export FASTERTOKENS_API_KEY=ft_live_…
```

## 2. Send a turn

```bash
curl https://api.fastertokens.com/v1/chat/completions \
  -H "Authorization: Bearer $FASTERTOKENS_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Session-Id: call-8841" \
  -d '{
    "model": "deepseek-v4.1-flash",
    "stream": true,
    "messages": [
      {"role": "system", "content": "You are a phone assistant. Reply in one short sentence."},
      {"role": "user", "content": "Hello?"}
    ]
  }'
```

The response is the standard stream of `chat.completion.chunk` events ending in `data: [DONE]`.

## 3. Use it from an OpenAI SDK

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.fastertokens.com/v1", api_key=os.environ["FASTERTOKENS_API_KEY"])

stream = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    messages=messages,
    tools=tools,
    stream=True,
    extra_headers={"X-Session-Id": call_id},
)
for chunk in stream:
    delta = chunk.choices[0].delta if chunk.choices else None
    if delta and delta.content:
        speak(delta.content)
```

```ts
import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'https://api.fastertokens.com/v1', apiKey: process.env.FASTERTOKENS_API_KEY });

const stream = await client.chat.completions.create(
  { model: 'deepseek-v4.1-flash', messages, tools, stream: true },
  { headers: { 'X-Session-Id': callId } },
);
```

## Send one session ID per call

Set the `X-Session-Id` header (or the `user` field) to an ID that stays the same for every turn of a call and differs between calls. The turns of a call are then served consistently, and later turns can reuse the cached prompt. Without it, every turn is treated as a new call.

## What to expect

- **Time to first token.** Every response carries `x-ttft-ms`: the time from your request reaching the endpoint to the first real output leaving it. Live percentiles are on the [benchmark page](https://fastertokens.com/benchmark).
- **Fail fast.** If no output is produced within 2 seconds, the request ends with HTTP 504 (`first_token_timeout`) before any stream starts, so your agent's fallback model can take the turn. You can change the deadline per request with `X-First-Token-Deadline-Ms` (300 to 10,000).
- **Sized for speech.** Reasoning is off. `max_tokens` defaults to 500 and is capped at 2,000.
- **Preview limits.** The preview is free. Each account has a monthly allowance of completions, shown in the dashboard; write to us if you need more.

Next: [voice agent setup](https://fastertokens.com/docs/voice-agents) and the [API reference](https://fastertokens.com/docs/api-reference).


---

# Voice agent setup

Any framework or platform that accepts an OpenAI-compatible model can use FasterTokens. You need three values:

| Setting | Value |
|---|---|
| Base URL | `https://api.fastertokens.com/v1` |
| API key | your `ft_live_…` key |
| Model | `deepseek-v4.1-flash` |

Keep your speech recognition, voices and telephony as they are.

## LiveKit Agents (Python)

The OpenAI plugin takes a base URL:

```python
import os
from livekit.agents import AgentSession
from livekit.plugins import openai

llm = openai.LLM(
    model="deepseek-v4.1-flash",
    base_url="https://api.fastertokens.com/v1",
    api_key=os.environ["FASTERTOKENS_API_KEY"],
)
session = AgentSession(llm=llm, stt=..., tts=...)
```

To give each call its own session, pass the call or room ID as the `user` value. The plugin forwards it with every request:

```python
llm = openai.LLM(model="deepseek-v4.1-flash", base_url="https://api.fastertokens.com/v1", api_key=key, user=call_id)
```

## Other frameworks and platforms

Use the framework's OpenAI-compatible LLM service or the platform's "custom model" or "custom LLM" setting, with the three values above. If the platform lets you add request headers, send `X-Session-Id` with the call ID; if it only lets you set OpenAI fields, set `user`.

## Keep a fallback

Querying several providers at once makes a slow first token rare; it does not make failure impossible. Configure a second model as a fallback for the turn:

- FasterTokens answers **HTTP 504** when no output was produced before the deadline (2 seconds by default), and **HTTP 502** when the request failed upstream. Both arrive before any part of a response, so the turn can be retried on another model without the caller hearing a restart.
- Set your framework's per-attempt timeout a little above the deadline you chose, so the 504 arrives before your own timeout fires.

In LiveKit Agents:

```python
from livekit.agents import llm as lk_llm

model = lk_llm.FallbackAdapter([fastertokens_llm, backup_llm], attempt_timeout=2.5, max_retry_per_llm=0)
```

## Tool calls

Tool definitions and streamed tool calls use the OpenAI format unchanged. On a turn that goes straight to a tool, time to first token is measured to the tool call.

## Checking latency yourself

Read `x-ttft-ms` from the response headers and log it next to your own measurement. The difference between the two is network time between your servers and the endpoint.


---

# API reference

Base URL: `https://api.fastertokens.com`. Authenticate with `Authorization: Bearer ft_live_…` (or `X-API-Key`).

## POST /v1/chat/completions

Create a completion. The request and response follow the OpenAI chat completions format.

### Request body

| Field | Notes |
|---|---|
| `model` | `deepseek-v4.1-flash`. May be omitted. |
| `messages` | Required. System, user, assistant and tool messages. |
| `tools`, `tool_choice`, `parallel_tool_calls` | Function tools, as in the OpenAI API. |
| `stream` | `true` for server-sent events. Recommended for voice. |
| `stream_options.include_usage` | Adds a final chunk with token usage. |
| `max_tokens` / `max_completion_tokens` | Default 500, capped at 2,000. |
| `temperature`, `top_p`, `stop`, `presence_penalty`, `frequency_penalty`, `seed`, `response_format` | Passed through. |
| `user` | Used as the session ID when no `X-Session-Id` header is sent. |

Other fields are ignored. Reasoning is always off. The request body may be up to 512 KB.

### Request headers

| Header | Notes |
|---|---|
| `X-Session-Id` | One ID per call. Turns that share it are served consistently and can reuse the cached prompt. |
| `X-First-Token-Deadline-Ms` | How long to wait for the first real output, 300 to 10,000. Default 2,000. |

### Response headers

| Header | Notes |
|---|---|
| `x-request-id` | The request's ID. Quote it when you write to us. |
| `x-ttft-ms` | Milliseconds from the request arriving to the first real output (text or a tool call). |

### Errors

Errors use the OpenAI shape: `{"error": {"message": "…", "type": "…", "code": "…"}}`.

| Status | `code` | Meaning |
|---|---|---|
| 400 | `invalid_request`, `invalid_json`, `model_not_found` | The request is malformed. |
| 401 | `unauthorized`, `invalid_api_key` | Missing, unknown or revoked key. |
| 413 | `request_too_large` | Body over 512 KB. |
| 429 | `rate_limited` | Over the key's requests per minute. |
| 429 | `too_many_concurrent_requests` | Over the key's requests in flight. |
| 429 | `monthly_limit_reached` | The account's monthly allowance is used up. |
| 502 | `upstream_failed` | The request failed upstream before producing output. Safe to retry or fall back. |
| 503 | `capacity_paused` | The preview's capacity for the day is used up. It reopens at 00:00 UTC. |
| 504 | `first_token_timeout` | No output was produced before the deadline. Safe to retry or fall back. |

502 and 504 are returned before any part of a response is sent. If a response is cut off after it has started, a streamed response ends with an `error` event instead of `[DONE]`.

## GET /v1/models

Lists the available model.

## GET /v1/usage

Your completions and tokens by day, your own time-to-first-token percentiles, and the month's allowance.

| Query | Notes |
|---|---|
| `from`, `to` | `YYYY-MM-DD`, UTC. Default: this month to date. |
| `key=this` | Only the key making the request. |

```json
{
  "from": "2026-11-01", "to": "2026-11-13", "requests": 1840,
  "ttft_ms": { "p50": 412, "p95": 655, "completions": 1836 },
  "month": { "used": 1840, "limit": 2000 },
  "days": [{ "date": "2026-11-12", "key_id": "key_…", "requests": 320, "prompt_tokens": 401220, "completion_tokens": 9180 }]
}
```

The numbers in this example are illustrative.

## GET /benchmark

Public, no key. The live benchmark summary shown on the [benchmark page](https://fastertokens.com/benchmark): percentiles overall, by scenario, by day and by hour, with a single provider's figures for comparison.

## Limits

| Limit | Default |
|---|---|
| Requests per minute, per key | 300 |
| Requests in flight, per key | 20 |
| Completions per month, per account (preview) | shown in the dashboard |

Write to [leko@manulabs.xyz](mailto:leko@manulabs.xyz) to raise a limit.

## What is stored

Timings, token counts, the request ID and the key used. Messages and model output are not written to our database or logs.


---

## FAQ

### What is FasterTokens?

An OpenAI-compatible chat completions endpoint for voice agents, serving DeepSeek V4.1 Flash. For every request it queries multiple inference providers simultaneously and optimizes for time to first token.

### Why does a large model stall or time out on the first token?

A model this size is expensive to serve, so providers typically run it on shared capacity. When that capacity is busy a request waits before the model starts, and the wait shows up as a first token that takes a second or more, or misses your timeout. Our public benchmark records how often that happens on a single provider.

### Why not call one provider directly?

Because every provider has those moments, and on a phone call a slow first token is dead air: the caller repeats themselves, or your agent times out and falls back to a weaker model. Querying several providers at once means one of them stalling rarely slows the turn.

### How is time to first token measured?

From the moment a request reaches our endpoint to the moment the first real output (text or a tool call) leaves it. A benchmark of typical voice turns runs around the clock and its results are public at /benchmark.

### Is there a latency guarantee?

Not yet. We are measuring a full week across every hour of the day before committing to a number in writing. The measurements are public while they run.

### Does it support streaming and tool calls?

Yes. Streaming responses use the standard server-sent event format, and tool calls stream as they do from the OpenAI API. Non-streaming requests are supported too.

### What does it cost?

The preview is free, with a monthly allowance of completions per account. Paid plans will be announced before any charge is made.

### Are prompts or completions stored?

No. We keep timings, token counts and request IDs for metering. Messages and model output are not written to our database or logs.