2,508 requests. 13.1 million tokens. Six people using the service. Those numbers gave us something more useful than a successful model demo: a record of what happened when our developers started using a self-hosted LLM through their normal tools.
This is the third article in our series on self-hosting open-weight models. Part 1 explains the fundamentals. Part 2 covers our Qwen and vLLM architecture. Here, we focus on what our request records and engineering notes actually tell us—and what they do not.
What did we learn from 2,508 self-hosted LLM requests?
From October 9 through November 3, 2026, Centizen recorded 2,508 requests and approximately 13.1 million tokens through our internal Qwen3 and vLLM service. Most requests came through developer-tool API keys rather than standalone chat. The logs recorded 97.1% of requests as successful, but that figure does not measure code correctness or developer productivity.
The issues we investigated were often at the boundaries between the editor, gateway, proxy and inference engine: tool schemas, token limits, parsing, streaming responses and routing. The practical lesson was to test the complete developer workflow, not just whether the model endpoint returns a response.
What we measured, and over what period
Our available records cover October 9 through November 3, 2026. They show 2,508 requests and approximately 13.1 million tokens. Of those requests, 2,436 were recorded as successful, which is 97.1%. The other 72 were not recorded as successful. We have not established a complete cause-by-cause breakdown for those 72 outcomes.
| Metric | Observed value |
|---|---|
| Observation period | October 9–November 3, 2026 |
| Total requests | 2,508 |
| Recorded successful requests | 2,436 |
| Other recorded outcomes | 72 |
| Recorded request success rate | 97.1% |
| Total recorded tokens | Approximately 13.1 million |
| Editor-key requests | 2,466 (98.3%) |
| Chat requests | 42 (1.7%) |
| Active requesters | 6 |
| Live API keys | 3 |
| Busiest recorded day | October 22: 2,057 requests |
These are descriptive request-log figures, not controlled benchmarks. They do not measure code quality, model accuracy or sustained capacity.
There were six active requesters and three live API keys in the records. The busiest recorded day was October 22, with 2,057 requests. A peak of six requests per second also appears in the notes. Neither number should be treated as a tested sustained capacity limit; a busy day and a controlled load test answer different questions.
October 22 accounted for approximately 82% of all recorded requests in this period. The traffic was therefore concentrated rather than evenly distributed across the dates. The records used for this article do not establish why that day was unusually busy.
This was mainly developer-tool traffic
Of the 2,508 requests, 2,466 (98.3%) were associated with editor API keys. Only 42 (1.7%) were recorded as chat requests. That split changed how we interpreted the service. We were not primarily evaluating a chatbot. We were learning how a model behaves when an editor packages source code, instructions, tool descriptions and conversation history into requests.
It also explains why a basic chat prompt was a weak acceptance test. A model might answer a direct question while an editor still fails to parse a tool call or rejects a streamed response.
A 97.1% success rate needs context
The recorded success rate is useful for spotting operational trends, but it is not a measure of code correctness, developer productivity or response quality. We did not run a controlled comparison against a hosted model, and we do not have a validated per-request latency distribution in these records.
Similarly, 13.1 million tokens is total recorded token volume. Dividing it by elapsed time would not give a meaningful decoding-speed benchmark. Input and output tokens, prompt length, concurrency and idle periods all affect the interpretation.
Where the integration work went
Some of the most time-consuming issues were not about whether Qwen could generate an answer. They appeared at the boundaries between the coding editor, our internal gateway, proxy routing and the vLLM response format.
We investigated editor tool-schema mismatches, arbitrary limits on tool-description lengths, incorrect token estimators, and parser settings that did not match the expected reasoning or tool-call format. Streaming introduced another problem: some clients expected different reasoning or tool-call delta fields than the server emitted.
We also encountered issues outside inference: a VS Code maxInputTokens setting, HTTPS private-network CORS behavior, and HAProxy path rewriting. A healthy inference endpoint did not guarantee that the actual editor workflow was healthy.
What changed in our testing approach
We started treating the client request as the unit of testing. Instead of asking only whether the server responds, we checked what the editor sends, what the gateway forwards, what vLLM streams, and what the editor can interpret.
Token budgeting is a good example. The text visible in the editor is only part of the request. Tool definitions and surrounding context consume tokens too. A rough estimate can be useful early, but near a hard limit we need an exact tokenization check against the serving model and tokenizer.
We also learned to confirm that a configuration change reached the running process. Editing a launch file is not proof that the active container picked up the new parser, context limit or routing setting.
What we can—and cannot—conclude about capacity
Our setup uses an on-premises GPU with 32 GB of VRAM, quantized Qwen3 models and vLLM. It was used for real internal requests. That supports a narrow conclusion: this arrangement was workable for the traffic we observed. It does not establish a universal GPU requirement or a concurrency promise for another team.
The source notes do not document a controlled throughput benchmark, p95 latency, measured cost per request, or a complete GPU utilization series. We also have no recorded out-of-memory, CUDA mismatch or model-load incidents in this evidence set; that is not proof such incidents could never occur.
The lessons we would carry into the next deployment
- Test the real editor early. A direct API call misses tool schemas, streaming and client token settings.
- Log enough to locate a failure. Separate editor, gateway, proxy and inference behavior without exposing private prompts or credentials.
- Record the exact runtime. Pin model revision, vLLM image digest, parser settings, GPU driver and launch command.
- Use precise metrics. Request success is not answer quality; token volume is not tokens per second.
- Load-test before expanding. Reproduce long-context and parallel workloads instead of extrapolating from a single busy day.
The most useful outcome of this work was not a headline benchmark. It was a clearer picture of the integration work required to make self-hosted inference useful to a development team. We now know which boundaries need repeatable tests—and which performance claims still need evidence.
Frequently asked questions
Can a self-hosted Qwen3 model support software development teams?
Centizen's records show that developers used a Qwen3/vLLM service through their coding tools. The deployment was workable for the requests we observed, but those records do not establish capacity or performance requirements for every development team.
What were the main problems with self-hosting Qwen3 using vLLM?
The documented issues were primarily integration-related: tool schemas, token estimation, tool-call and reasoning parser settings, streaming response formats, HTTPS access and proxy routing. We cannot attribute every unsuccessful request to a particular cause.
Does a 97.1% request success rate mean the model produced correct code?
No. The 97.1% figure describes how the service recorded request outcomes. It does not measure code correctness, response quality, development speed or task completion.
How should teams test a self-hosted LLM for coding tools?
Use real editor requests, not only direct API calls. Include tool calls, long prompts, streamed responses, token limits, and parallel workloads. Trace each request from the editor through the gateway and inference engine, then check what the editor actually accepted.
Technical references
- vLLM documentation — serving features, API behavior and configuration guidance.
- Hugging Face model repositories — model artifacts, revisions and tokenizer details.
Centizen request counts and integration observations in this article come from our internal records, not from these external references.
Explore the series: Start with Part 1: Self-Hosting Open-Source LLMs, then read Part 2: Our Qwen/vLLM architecture and engineering decisions. This third article covers our observed request activity and the integration lessons behind it.
Our services:
- Staffing: Contract, contract-to-hire, direct hire, remote global hiring, SOW projects, and managed services.
- Remote hiring: Hire full-time IT professionals from our India-based talent network.
- Custom software development: Web/Mobile Development, UI/UX Design, QA & Automation, API Integration, DevOps, and Product Development.
Our products:
- ZenBasket: A customizable ecommerce platform.
- Zenyo payroll: Automated payroll processing for India.
- Zenyo workforce: Streamlined HR and productivity tools.