llmRouter

A local OpenAI-compatible gateway that spreads requests across free model providers, tracks their limits and falls back on its own.

llmRouter is the lab's "API". It presents one OpenAI-compatible endpoint, and behind it spreads requests across many free language-model providers. Applications such as JobHunter talk to llmRouter and never worry about which provider is free, which has hit its limit, or which is down.

What it does

• One endpoint. Clients use standard OpenAI-style chat, vision and image routes, and a single API key.
• Fan-out. Requests are distributed across Google AI Studio, NVIDIA NIM, Cerebras, GitHub Models, Groq, Cloudflare Workers AI, OpenRouter and LLM7.io.
• Limit tracking. Per-provider and per-model rate limits are kept in SQLite.
• Pacing. Daily quotas are spread across the day instead of being spent in the first hour.
• Fallback. If a provider fails or is rate-limited, the next one answers.

The proxy refuses to start without its own API key, so clients must authenticate even on a private network.

Web providers

Two optional providers reach ChatGPT and Gemini through their web apps using browser cookies, with no API key. They sit below every API provider as a last resort, and are skipped quietly when the extra package or the cookies are missing. Cookies are refreshed automatically when authentication fails.

Image generation

Text-to-image and image editing are served by providers that support them. Routing is by capability: the router works out from the request whether it needs a plain image, an edit or a masked edit, and only offers models that can do that job. A masked edit is never sent to a model that cannot take a mask.

Images come back inline by default. Alternatively, the proxy keeps the bytes briefly and returns a short-lived link.

Health

The health endpoint reports each provider's state: enabled, running, available, rate-limited, or disabled and why. The status page shows the gateway as degraded if its own health report is not healthy.

Where it sits

llmRouter listens on loopback. Docker applications reach it through a small relay on the Docker bridge, and the A06 node through an authenticated proxy, so the gateway itself is never exposed directly.