Source proof for cloudscale.ch evaluates self-hosted coding LLMs on its own GPU instances with vLLM and LiteLLM

…f-hosting with hosted models and public providers like Anthropic’s Claude Code. We spent several weeks evaluating this setup. The short answer: self-hosting works, but you inherit complexity that hosted pr…

…─────┬────────┘ │ ┌──────▼────────┐ │ GPU │ │ RTX 6000 96GB │ └───────────────┘ Caddy handles TLS termination. LiteLLM provides an API gateway with model routing. vLLM loads and serves the model using the underlying GPU. Why LiteLLM LiteLLM gives you a single API endpoint that can route requests acr…

…but you inherit complexity that hosted providers hide from you. The test setup cloudscale.ch offers GPU instances equipped with NVIDIA RTX PRO 6000 Max-Q GPUs with 96GB VRAM each. Up to four GPUs can be attached to a single VM. Our setup consisted of one VM w…

…urrent requests. We haven't tested it yet, but it's high on our list. Our test setup was minimal: a single engineer comparing results between the self-hosted models and Claude Opus 4.6/4.7 using Claude Code. Towards the end of our testing period, we briefly tested concurrency, but only…

…ur testing period, we briefly tested concurrency, but only to a limited extent. A broader, production-like test is still pending. Model evaluation Initially we tested Qwen2.5-Coder-32B and deepseek-coder-33b-a…

…ases, the number of users and expected context window are important parameters. For cloudscale.ch, we have not yet reached a final conclusion. What is clear, however, is that self-hosting coding LLMs is viable but introduc…

…. Zurück zur Übersicht Starte noch heute E-Mail Adresse Konto erstellen Kontakt cloudscale.ch AG Neugasse 6 CH-8005 Zürich T +41 44 55 222 55 F +41 44 55 222 56 / Sitemap Angebot Preise Marktplatz Siche…

…. Zurück zur Übersicht Starte noch heute E-Mail Adresse Konto erstellen Kontakt cloudscale.ch AG Neugasse 6 CH-8005 Zürich T +41 44 55 222 55 F +41 44 55 222 56 / Sitemap Angebot Preise Marktplatz Siche…

…but you inherit complexity that hosted providers hide from you. The test setup cloudscale.ch offers GPU instances equipped with NVIDIA RTX PRO 6000 Max-Q GPUs with 96GB VRAM each. Up to four GPUs can be attached to a single VM. Our setup consisted of one VM w…

…test setup Model evaluation The time-consuming part: Configuration & Iterations In today’s world, engineers and businesses increasingly expect to develop software with the help of coding agents. There is a wide range of agents and models available, each with its own trade-o…

…ffs. At cloudscale, we care deeply about data privacy and digital sovereignty . We have our own GPU instances which we fully control. When discussing coding agents, it therefore felt natural to compare self-hostin…

…and digital sovereignty . We have our own GPU instances which we fully control. When discussing coding agents, it therefore felt natural to compare self-hosting with hosted models and public providers like Anthropic’s Claude Code. We spent several weeks evaluating this setup. The short answer: self-hosting wo…

…Max-Q GPUs with 96GB VRAM each. Up to four GPUs can be attached to a single VM. Our setup consisted of one VM with one GPU attached: ┌───────────────┐ │ Clients │ │ (opencode, │ │ agents, …) │ └──────┬────────┘…

…ited extent. A broader, production-like test is still pending. Model evaluation Initially we tested Qwen2.5-Coder-32B and deepseek-coder-33b-awq. This was not sufficient for our use case. Partly because our setup was still su…

…est period, but also because newer models are just much better than older ones. When Gemma 4 was released, we immediately switched to it (first gemma-4-31B-it, then gemma-4-26B-A4B-it). The improvement was night and…

…laude Code, it still lagged significantly behind when it comes to code quality. Qwen3.6-35B was finally the model which gave us confidence that self-hosting is viable. Coding quality is quite good, and reasoning ability is solid as well. Compared…

…it's not quite there yet, but it's actually usable. To compare more, we also configured LiteLLM to proxy models from Infomaniak: Qwen3-VL-235B, GPT-OSS-120B, Kimi-K2.5. We chose Infomaniak because it is a Swiss provider that hosts these models on i…

…choice, agent compatibility turned out to be a bigger constraint than expected. Opencode worked very well from the get-go. It's highly customizable and easy to configure different models for differ…

…like exposing model reasoning are not consistently supported with all models). What you get with self-hosted is full control, data privacy, and predictable cost. But it requires time to tune, debug, and maintain. Case in point: during our testing period, we upgraded vLLM and LiteLLM almost d…

…nges pricing or model behavior, your engineering workflow is directly affected. To sum up, self-hosting is viable, but not plug-and-play. You inherit the complexity that hosted providers hide. This partially also appl…

…test setup Model evaluation The time-consuming part: Configuration & Iterations In today’s world, engineers and businesses increasingly expect to develop software with the help of coding agents. There is a wide range of agents and models available, each with its own trade-o…

…─────┬────────┘ │ ┌──────▼────────┐ │ GPU │ │ RTX 6000 96GB │ └───────────────┘ Caddy handles TLS termination. LiteLLM provides an API gateway with model routing. vLLM loads and serves the model using the underlying GPU. Why LiteLLM LiteLLM gives you a single API endpoint that can route requests acr…

…urrent requests. We haven't tested it yet, but it's high on our list. Our test setup was minimal: a single engineer comparing results between the self-hosted models and Claude Opus 4.6/4.7 using Claude Code. Towards the end of our testing period, we briefly tested concurrency, but only…

…nges pricing or model behavior, your engineering workflow is directly affected. To sum up, self-hosting is viable, but not plug-and-play. You inherit the complexity that hosted providers hide. This partially also appl…

…f-hosting with hosted models and public providers like Anthropic’s Claude Code. We spent several weeks evaluating this setup. The short answer: self-hosting works, but you inherit complexity that hosted pr…

…test setup Model evaluation The time-consuming part: Configuration & Iterations In today’s world, engineers and businesses increasingly expect to develop software with the help of coding agents. There is a wide range of agents and models available, each with its own trade-o…