Native Go Routing Engine

Janus API Gateway

A native Go LLM router built to replace Python proxy stacks. One compiled binary, goroutine multiplexing, and no runtime dependencies to install or break.

Concurrency

No Python GIL

Compiled Go goroutines bypass Python's Global Interpreter Lock (GIL), routing many concurrent prompts at once.

Zero Bloat

Single Static Binary

No virtual environments, no pip conflicts, no CUDA version hell. Copy one file to the machine and run it.

Universal Endpoint

OpenAI Compatible

Drop-in replacement for standard /v1/chat/completions routes. Compatible with Ollama, Vulkan runners, OpenRouter, and Claude.

Hardware Acceleration

Vulkan, Any GPU

Hardware-agnostic Vulkan backend — runs models on AMD, Intel, or NVIDIA GPUs, or straight on CPU and system RAM.

Architecture Comparison

What you actually ship, and what you have to keep working.

janus — deployment shape
Ships as      one compiled binary — copy it to the box and run it
Installs       nothing — no venv, no pip tree, no CUDA toolkit to match
Speaks        /v1/chat/completions — anything OpenAI-compatible just points at it
Routes to      local Vulkan & Ollama runners, OpenRouter, Claude — same endpoint
Runs on       AMD / Intel / NVIDIA GPU, or CPU + system RAM — Vulkan, not vendor-locked

Switching From Something Else?

If you're already running local models, here's what changes with Janus.

vs Ollama

No Background Daemon

Ollama runs as a system service that manages its own model format and storage. Janus runs when you launch it and stops when you close it. Point it at any standard .gguf file — no model conversion, no separate model pull system.

vs LM Studio

No Electron App

LM Studio is a desktop GUI built on Electron. Janus is a headless server you control with HTTP calls. If you're scripting, piping output, or connecting Cursor or Cline to a local model, Janus gives you the API without the app.

vs Python Stacks

No CUDA Lock-In

vLLM, text-generation-inference, and most Python inference servers require NVIDIA GPUs and CUDA. Janus uses Vulkan — your AMD, Intel, or NVIDIA GPU works the same. No driver version matching, no torch reinstalls.

vs Running llama.cpp Directly

OpenAI API Built In

Raw llama.cpp gives you a CLI or a basic HTTP server. Janus wraps it in a proper OpenAI-compatible API with streaming, chat templates auto-detected from the model file, and hot-swap so you can change models without restarting.

Common Questions

Can I use this with Cursor or Cline?

Yes. Set your base URL to http://127.0.0.1:8990/v1 and leave the API key blank. Any tool that speaks the OpenAI API works — Cursor, Cline, Continue, custom scripts, anything.

Does it only run local models?

No. Set INFERENCE_BACKEND=openrouter in your config and Janus routes to cloud models through OpenRouter — same API, same endpoint. Switch between local and cloud by changing one line.

What GPUs work?

Anything with Vulkan support — AMD, Intel, and NVIDIA. No CUDA required. CPU fallback is available but significantly slower. GPU is recommended for usable generation speeds.

Is it free?

Yes. Janus is MIT licensed and open source. View on GitHub.