Best Speculative Decoding for Uncensored LLM Use
Speculative decoding is a server-side speed technique. This article explains how it works, which variants fit an uncensored model, and what you can do from the client side of an API.
Updated

Cheap AI APIs
Key points
- Speculative decoding drafts several tokens cheaply, then checks them in one pass of the main model.
- With correct sampling, the output distribution matches ordinary decoding; only the speed changes.
- On a hosted API the setup is the provider's choice, not a request parameter.
- This site does not publish speed figures; measure on your own prompts with the script below.
Short answer
If you are asking about the best speculative decoding for uncensored llm deployments, the answer depends on who holds the server. When you run the model yourself, a draft model or a head trained for that exact model usually wins; when you call a hosted API, you cannot choose at all, because speculative decoding is a property of the inference server and not a field in the request. No parameter of POST /v1/chat/completions switches it on.
This API does not state which decoding method sits behind it, and this article will not guess. What follows is the general technique, how to pick among methods if you self-host, and what you can measure and tune from the outside.
How speculative decoding works
A language model normally writes one token per forward pass, and each pass has to read the model weights from memory. That makes generation limited by memory bandwidth rather than arithmetic. Speculative decoding exploits the idle arithmetic.
- A cheap draft mechanism proposes the next few tokens, say four.
- The full model scores all four positions in a single forward pass.
- It accepts the longest prefix that matches what it would have produced, and corrects the first mismatch.
- Generation continues from there.
With the published rejection-sampling scheme, the accepted tokens follow the same distribution as plain sampling from the main model. So quality does not change by design. You get several tokens for the price of about one pass when the draft guesses well, and roughly one token per pass when it does not. The speed-up is the average number of accepted tokens, which is called the acceptance rate in the literature.
Methods and when each fits
| Method | How it drafts | Good fit |
|---|---|---|
| Small draft model | A smaller model from the same family proposes tokens | General chat, if a close small sibling exists |
| Prompt lookup / n-gram | Copies spans that already appear in the prompt | Editing, summarising, code rewriting, retrieval answers |
| Extra decoding heads (Medusa-style) | Added heads predict several future tokens | Self-hosting with training budget |
| Feature-level drafting (EAGLE-style) | A light module predicts from the model's own hidden states | Self-hosting where a trained module exists |
Popular inference engines, including vLLM and llama.cpp, support speculative decoding in some form, though options and maturity differ by version, so check the documentation of the release you run.
No method is best in the abstract. Prompt lookup costs nothing to set up and shines when the output repeats the input. A draft model helps most when it agrees with the target on style. Trained heads give steady gains on the model they were trained for.
Why acceptance rate decides the speed-up
A simple model shows the effect. If the drafter proposes k tokens and each one is accepted with probability p, independently, the main model produces on average (1 - p^(k+1)) / (1 - p) tokens per pass. This ignores the time spent drafting, so real gains are smaller. With k = 4:
| Acceptance p | Tokens per pass | Reading |
|---|---|---|
| 0.8 | 3.36 | Draft agrees well; large gain |
| 0.5 | 1.94 | Modest gain, easily eaten by draft cost |
| 0.3 | 1.43 | Little gain; may be slower overall |
The numbers explain why matching the drafter to the target matters more than picking a famous method. A mediocre drafter at p = 0.3 barely beats plain decoding, while a well-matched one at p = 0.8 produces more than three tokens per pass.
What changes for an uncensored model
The method itself is indifferent to content policy. The practical catch is draft agreement. Many small models are released with refusal behaviour baked in. An uncensored target model writes things such a draft would never propose, or would hedge. When the draft keeps predicting a disclaimer and the target keeps writing the scene, the acceptance rate falls, and the extra drafting work can cost more time than it saves.
That gives a simple rule for self-hosters. Use a draft that was tuned in the same direction as the target, or a method that needs no separate model: prompt lookup, or heads and modules trained on the target itself. Verify with a trial run on your own prompts, because creative writing at high temperature is the hardest case for any drafter. Randomness lowers agreement, and fiction tends to run hot.
For the API on this site, the model is one open-weight model run by an independent provider, tuned to answer without content refusals for lawful adult use. Sexual content involving minors is not allowed under any setting.
What you can control as an API user
You cannot toggle the drafter, but several request settings affect how fast a reply feels.
- Stream the response. With
stream: truethe first words arrive while the rest is still being generated. Perceived speed improves regardless of server tricks. Remember a stream holds its concurrency slot until it ends; a full key allows 8 at once. - Limit output.
max_tokensis optional, but a cap bounds the worst-case wait and cost. - Lower temperature. If your server does use speculation, low-temperature output is easier to predict, so it tends to go faster. Treat that as something to test, not a promise.
- Keep prompts short. Less input to process before the first token appears.
- Parallelise within limits. Up to 8 concurrent requests and 300 per minute on a full key improve throughput of a batch, even if one reply is not faster.
Speed does not change what you pay. Billing is $0.15 per 1M input tokens and $0.50 per 1M output tokens either way, and failed requests are not charged. See the pricing page for the details.
Measure it yourself
Do not trust anyone's speed number, including a provider's, until you reproduce it with your own prompts. This script times a non-streamed request and divides completion tokens by wall time. It runs the same prompt at two temperatures with a fixed seed.
import time
from openai import OpenAI
client = OpenAI(base_url="https://api.cheapaiapis.com/v1", api_key="YOUR_KEY")
def run(prompt, temperature):
t0 = time.time()
r = client.chat.completions.create(
model="uncensored",
messages=[{"role": "user", "content": prompt}],
temperature=temperature,
max_tokens=400,
seed=7,
)
dt = time.time() - t0
n = r.usage.completion_tokens
return n, dt, n / dt
for temp in (0.0, 0.8):
n, dt, tps = run("Write a short scene set in a lighthouse.", temp)
print(f"temp={temp} tokens={n} seconds={dt:.1f} tokens/s={tps:.1f}")Run it several times at different hours and take the median. The figure includes network time and queueing, so it is an end-to-end number, which is what your users experience. If you want time to first token, switch to stream=True and record when the first chunk arrives. The quickstart shows the streaming call.
You can use the trial credit for this. One click on the API key page gives $0.50, valid for 7 days, no card. The script above uses at most 400 output tokens per run, so ten runs of 400 output and 20 input tokens cost 10 x ($0.000003 + $0.0002) = $0.00203, a fifth of a cent.
If you plan to self-host
Start without speculation and record a baseline. Then add prompt lookup if your workload copies text from the prompt. Next try a draft model that shares the target's tokenizer and tuning direction. Measure tokens per second and the share of drafted tokens accepted, at your real temperature. Stop when the gain stops being worth the extra memory, since a draft model also needs GPU space.
If self-hosting is more work than you want, a hosted endpoint with per-token billing removes the question. The article on cheap API myths and facts covers what to check before trusting any low price.
More guides
Questions and answers
Read the docsDoes speculative decoding change the quality of answers?
Not when implemented with the standard acceptance rule, because accepted tokens follow the main model's distribution. It changes speed only.
Can I enable speculative decoding with a request parameter?
No. It is configured on the inference server. Nothing in the chat completions request turns it on for this API.
Is it faster at temperature 0?
Often, since predictions are easier to match, but it varies by method and prompt. Time your own workload.
Which method is best for roleplay and fiction?
Creative text at high temperature is hard to draft. Prompt lookup helps little there, so a draft or heads trained on the same model is the better bet if you self-host.
Your key is one form away
Create an account, copy the key, change the base URL. That is the whole setup.
https://api.cheapaiapis.com/v1