VS Code, with LM Studio behind it.
LM Studio is a desktop app for finding, downloading and running models with a graphical interface. Its OpenAI-compatible local server can serve all three of twinny's jobs: chat, completion and embeddings. twinny asks the server what it has loaded and fills in the model for each.
- 01
In LM Studio, download an instruct coder model for chat, a small base coder model for completion, and an embedding model if you want the index. Start the server on the Developer tab; the default port is 1234.
- 02
In twinny, Add provider, then LM Studio. It lists the models the server reports and picks the first match for each job.
- 03
Type. If a request says the model is not found, load it in LM Studio or switch on just-in-time loading.
Several models at once
LM Studio can keep more than one model loaded when memory allows, so chat and completion can be different models on the same server.
Another machine
Turn on Serve on Local Network in LM Studio and point the provider at that machine's address.
Context per model
The context length is set per model in LM Studio's settings. Raise it for code review and long chats.
One gateway for the team
twinny-server sits on the machine with the models and gives every developer a key, usage per person, policy and an admin page. Free for 5 developers, $6 a seat after that, 30-day trial.
- What is the model name?
- The identifier LM Studio's server reports, shown on the Developer tab; usually the repository name in lower case.
- Which routes does twinny use?
- Chat on /v1, completion on /v1/completions with a fill-in-the-middle prompt, embeddings on /v1/embeddings.
- Does anything leave my machine?
- Only the requests to the LM Studio address you configured. No telemetry, no account.
Everything else is on the home page: features, the team gateway, security and pricing.