Serve your model from the cheapest GPU that is free right now.
Deploy an LLM, a diffusion model, or anything in a container. Surfra keeps one endpoint in front of it, starts GPUs on your clouds when traffic arrives, and stops them when it stops.
One endpoint, whichever GPU is free.
What you want from serverless inference is a single URL that answers, and a bill that falls when the traffic does. Surfra is the URL. The GPUs stay in accounts you already have, and each cloud bills you at its own price. You list machines in the order you want them tried. A new replica takes the first row with room, so a sold-out card is a fallback instead of a stuck queue.
If the model already runs in vLLM, SGLang, TGI, or Ollama, leave the server as it is. The vLLM example puts the Surfra supervisor in front of the image you already use. The same Dockerfile pattern works for the other three.
FROM vllm/vllm-openai:latest RUN curl -fsSL https://surfra.io/install.sh | sh ENTRYPOINT ["surfra", "worker"]
The deploy names the server, the port, and the path Surfra should call. Salad RTX 4090 is first. RunPod RTX 4090 takes the replica when Salad has no room. Callers keep posting to the same app either way. Per-provider limits, set on a row, keep a base on the cheap cloud and send only the spike to the next one, so a traffic bump does not move the whole app onto the expensive card.
The other way in is a Python handler: load the weights once in a warmup, then return JSON. The Batch example is that path, and it takes several prompts in one forward pass. An HTTP app is the one below.
import surfra surfra.App("qwen").deploy( image="ghcr.io/you/vllm-surfra:1", entrypoint="vllm serve Qwen/Qwen3-0.6B --host 127.0.0.1 --port 8000", runtime={ "engine": "http", "port": 8000, "path": "/v1/chat/completions", "ready_path": "/health", "ready_timeout": 1800, }, autoscale={ "max_replicas": 3, "idle_timeout_sec": 600, "execution_timeout_sec": 600, "providers": [ {"provider": "salad", "gpu": "rtx4090", "priority": 1}, {"provider": "runpod", "gpu": "rtx4090", "priority": 2}, ], }, )
Call it the way you already call OpenAI
An HTTP app also answers OpenAI-compatible calls, so a client that already speaks that API changes its base URL and its key. Nothing else in the application has to know which cloud answered.
import os from openai import OpenAI client = OpenAI(base_url="https://api.surfra.io/openai/qwen/v1", api_key=os.environ["SURFRA_API_KEY"]) reply = client.chat.completions.create( model="Qwen/Qwen3-0.6B", messages=[{"role": "user", "content": "Say hi"}], ) print(reply.choices[0].message.content)
Quiet hours
Replicas stop after idle_timeout_sec. A quiet app goes to zero machines, and the cloud stops billing them. The next call waits while a replica starts. How long that wait is depends on the cloud and on your image. Surfra does not publish one cold-start number, because a single figure would be wrong for most of the clouds on the list.
If the first token has to be fast, set a minimum on that row and leave one warm. That machine stays up, and you pay the cloud for it the whole time. A fine-tune or a million embeddings does not belong in this queue. Those go to a job or a batch app. Mixing them in is how a batch sits in front of a person waiting on a reply.
Keep going.
Common questions
Can I deploy vLLM on Surfra?
Yes. Run vLLM as an HTTP app behind the Surfra supervisor. It also gets an OpenAI-compatible endpoint.
Does it scale to zero?
Yes. Replicas stop after the idle timeout you set. Keep a minimum if you want one warm.
Run it on your own clouds.
Surfra is invite only while we test it. Tell us what you want to run and we will send an invite.
Request access