Which template does completion use?

twinny detects the fill-in-the-middle template from the model name (Qwen, CodeLlama, DeepSeek, StarCoder and others). If a file is renamed, choose the template in the provider.

Can one llama-server do everything?

It can serve chat and completion for one model. For a smaller completion model, run a second server on another port. Embeddings need --embedding on the server that answers them.

Does twinny support the llama.cpp infill endpoint?

twinny renders the fill-in-the-middle prompt itself and sends it to the completion route, so any llama-server build works.

twinny with

VS Code, straight onto llama-server.

llama.cpp is the engine behind most local servers. Running its own llama-server gives you full control of the model file, context size, GPU offload and threads, with the least overhead. twinny talks to it directly: one server per model, on whatever ports you choose.

install for VS Code llama.cpp in the docs live demo

three steps

01
Download a GGUF and start llama-server with the context size and GPU offload you want. Add --embedding if the same server should answer embedding requests.
02
In twinny, Add provider, then llama.cpp. Set the port for each job; a second llama-server on another port can serve a smaller completion model.
03
Type. Completions stream as ghost text; the fill-in-the-middle template is picked from the model name, or by hand for a renamed file.

Nothing between you and the model

No daemon, no model manager: the GGUF you chose, the flags you set. twinny sends the prompt and shows the answer.

One server per job

A 7B instruct model on one port for chat and edits, a 1.5B base model on another for completion. Each provider in twinny points at its own port.

Every feature

Autocomplete, chat with @-mentions of files, symbols, the diff and the terminal, inline edit as a reviewable diff, code review of a branch or a pull request.

One gateway for the team

twinny-server sits on the machine with the models and gives every developer a key, usage per person, policy and an admin page. Free for 5 developers, $6 a seat after that, 30-day trial.

questions

Which template does completion use?: twinny detects the fill-in-the-middle template from the model name (Qwen, CodeLlama, DeepSeek, StarCoder and others). If a file is renamed, choose the template in the provider.
Can one llama-server do everything?: It can serve chat and completion for one model. For a smaller completion model, run a second server on another port. Embeddings need --embedding on the server that answers them.
Does twinny support the llama.cpp infill endpoint?: twinny renders the fill-in-the-middle prompt itself and sends it to the completion route, so any llama-server build works.

Everything else is on the home page: features, the team gateway, security and pricing.