Stop Overdrafts on
LLM API Streams
docker run -p 127.0.0.1:8080:8080 yodsarun/kilovolt-proxy:latestRun Kilovolt locally, then open the URL printed in your terminal.
No Git clone, Cargo, Docker Compose, or .env is required to evaluate the gateway.
Calculated Budget Circuit Breaker
Atomically reserves estimated prompt cost against project and user limits, then stops before forwarding an output increment that would exceed either limit.
Gemini SSE Translation
Experimental streaming-only translation for supported Gemini candidate text into OpenAI-shaped SSE chunks.
Bounded SSE Processing
Reconstructs UTF-8 SSE events across arbitrary network chunks and enforces an explicit maximum buffered frame size.
Measured locally with reproducible evidence
Apple M4, macOS arm64, release build, deterministic loopback mock. These are one-machine observations, not universal guarantees.
| Dimension | Direct Local Mock | Through Kilovolt ⚡ |
|---|---|---|
| Idle RSS across 3 runs | — | 10,208–10,256 KiB |
| JSON total p50, concurrency 1 | 0.107–0.115 ms | 0.149–0.152 ms |
| Streaming TTFB p50, concurrency 100 | 2.977–3.186 ms | 2.743–3.528 ms |
| 5,000-event stream total p50 | 2.484–2.663 ms | 21.233–21.371 ms |
| Normal-workload RSS after warm-up | — | about 61.5–64.5 MiB |
Methodology, raw JSON, limitations, and the dirty source-state disclosure are in the repository benchmark documentation.
# Point your client directly to local Kilovolt proxy gateway
curl -i -N -X POST http://127.0.0.1:8080/v1/chat/completions \
-H "Authorization: Bearer sk-proj-your-key" \
-H "X-User-ID: developer_alice" \
-d '{"model": "gemini-1.5-flash", "messages": [{"role": "user", "content": "Hi!"}], "stream": true}'