Skip to content
FasterTokens
View as Markdown

Voice agent setup

Any framework or platform that accepts an OpenAI-compatible model can use FasterTokens. You need three values:

Setting Value
Base URL https://api.fastertokens.com/v1
API key your ft_live_… key
Model deepseek-v4.1-flash

Keep your speech recognition, voices and telephony as they are.

LiveKit Agents (Python)

The OpenAI plugin takes a base URL:

import os
from livekit.agents import AgentSession
from livekit.plugins import openai

llm = openai.LLM(
    model="deepseek-v4.1-flash",
    base_url="https://api.fastertokens.com/v1",
    api_key=os.environ["FASTERTOKENS_API_KEY"],
)
session = AgentSession(llm=llm, stt=..., tts=...)

To give each call its own session, pass the call or room ID as the user value. The plugin forwards it with every request:

llm = openai.LLM(model="deepseek-v4.1-flash", base_url="https://api.fastertokens.com/v1", api_key=key, user=call_id)

Other frameworks and platforms

Use the framework's OpenAI-compatible LLM service or the platform's "custom model" or "custom LLM" setting, with the three values above. If the platform lets you add request headers, send X-Session-Id with the call ID; if it only lets you set OpenAI fields, set user.

Keep a fallback

Querying several providers at once makes a slow first token rare; it does not make failure impossible. Configure a second model as a fallback for the turn:

  • FasterTokens answers HTTP 504 when no output was produced before the deadline (2 seconds by default), and HTTP 502 when the request failed upstream. Both arrive before any part of a response, so the turn can be retried on another model without the caller hearing a restart.
  • Set your framework's per-attempt timeout a little above the deadline you chose, so the 504 arrives before your own timeout fires.

In LiveKit Agents:

from livekit.agents import llm as lk_llm

model = lk_llm.FallbackAdapter([fastertokens_llm, backup_llm], attempt_timeout=2.5, max_retry_per_llm=0)

Tool calls

Tool definitions and streamed tool calls use the OpenAI format unchanged. On a turn that goes straight to a tool, time to first token is measured to the tool call.

Checking latency yourself

Read x-ttft-ms from the response headers and log it next to your own measurement. The difference between the two is network time between your servers and the endpoint.