A native Go LLM router built to replace Python proxy stacks. One compiled binary, goroutine multiplexing, and no runtime dependencies to install or break.
Compiled Go goroutines bypass Python's Global Interpreter Lock (GIL), routing many concurrent prompts at once.
No virtual environments, no pip conflicts, no CUDA version hell. Copy one file to the machine and run it.
Drop-in replacement for standard /v1/chat/completions routes. Compatible with Ollama, Vulkan runners, OpenRouter, and Claude.
Hardware-agnostic Vulkan backend — runs models on AMD, Intel, or NVIDIA GPUs, or straight on CPU and system RAM.
What you actually ship, and what you have to keep working.
/v1/chat/completions — anything OpenAI-compatible just points at itIf you're already running local models, here's what changes with Janus.
Ollama runs as a system service that manages its own model format and storage. Janus runs when you launch it and stops when you close it. Point it at any standard .gguf file — no model conversion, no separate model pull system.
LM Studio is a desktop GUI built on Electron. Janus is a headless server you control with HTTP calls. If you're scripting, piping output, or connecting Cursor or Cline to a local model, Janus gives you the API without the app.
vLLM, text-generation-inference, and most Python inference servers require NVIDIA GPUs and CUDA. Janus uses Vulkan — your AMD, Intel, or NVIDIA GPU works the same. No driver version matching, no torch reinstalls.
Raw llama.cpp gives you a CLI or a basic HTTP server. Janus wraps it in a proper OpenAI-compatible API with streaming, chat templates auto-detected from the model file, and hot-swap so you can change models without restarting.
Yes. Set your base URL to http://127.0.0.1:8990/v1 and leave the API key blank. Any tool that speaks the OpenAI API works — Cursor, Cline, Continue, custom scripts, anything.
No. Set INFERENCE_BACKEND=openrouter in your config and Janus routes to cloud models through OpenRouter — same API, same endpoint. Switch between local and cloud by changing one line.
Anything with Vulkan support — AMD, Intel, and NVIDIA. No CUDA required. CPU fallback is available but significantly slower. GPU is recommended for usable generation speeds.
Yes. Janus is MIT licensed and open source. View on GitHub.