OmniRoute: the open-source AI gateway with 28,000 stars, and what it asks in return

On 30 June 2026 a repository few people had heard of reached the top of GitHub Trending. It is called OmniRoute, it is written in TypeScript, it ships under an MIT licence, and it now carries roughly 28,500 stars and 4,400 forks. The README opens with a slogan built to stick, a promise of coding that simply keeps going. Below it sits the operational substance: a single local endpoint, 290 declared model providers (90 of them with a free tier), 500 models, and routing that moves to the next provider the moment the first one runs out of quota.

Anyone who spends the day inside Claude Code, Cursor, Codex or Cline gets the appeal in three seconds. The rest of the picture deserves the same attention.

Hundreds of strands converging into a single bundle: the architectural promise of a gateway, and the exact place where the risk concentrates.

One endpoint instead of thirty keys

The software runs on the developer’s own machine and exposes an OpenAI-compatible address, http://localhost:20128/v1. Development tools point at it and stop caring which model answers: the router decides, using twelve live signals (provider health, remaining quota, cost, latency, success rate) across nineteen configurable strategies, from fixed priority to lowest cost, from round-robin to a panel-with-judge setup.

Around the router sits everything else: a compression layer that rewrites prompts to spend fewer tokens, a built-in MCP server, a desktop app and a PWA, three-layer resilience with a per-provider circuit breaker and per-model lockout. That is a remarkable amount of functional surface for a young project, which is both its strength and the first clue worth following.

Where the free tokens come from

The figure that travels furthest is “1.5 billion free tokens per month”. The method behind it is stated openly: the sum of the documented free tiers across more than forty provider pools. It is a theoretical sum. To reach it, a user opens an account with each service, keeps each one alive, and accepts that free tiers change terms whenever the vendors decide. A hands-on review published on 20 July counts 11 providers that answer before any registration, out of more than 230 available.

There is a further layer, visible in the project’s own public discussions. Some “free” providers work by reading session cookies from web interfaces: DeepSeek Web, Qwen Web Free, LMarena Free. In practice the gateway presents itself to a service designed for a human browser while impersonating that browser. The repository also documents TLS fingerprint spoofing (JA3/JA4) and a transparent MITM proxy for tools that ignore proxy settings: two technologies born to defeat anti-bot systems.

The sharpest independent analysis, written by German developer Florian Gahn, lists 19 providers whose terms explicitly forbid proxy access, Fireworks, NLPCloud and Modal among them, and recalls that Google Antigravity forbids third-party software access altogether. Contractual liability stays entirely with the account of whoever installs the thing.

The security chapter

In May, Socket.dev blocked the npm package at version 3.8.5 with a supply-chain score of 48 out of 100. Issue #2863, still open, lists the reasons in six points: installation of a root certificate on Windows, macOS and Linux; DNS and hosts-file manipulation; a bundled MITM server; reading provider tokens from the system keychain via keytar; elevated PowerShell execution with ExecutionPolicy Bypass; synchronisation of accessToken and refreshToken to a configurable cloud endpoint. The maintainer fixed two genuine vulnerabilities in 3.8.6, a credential-overwrite flaw in Cloud Sync and a defect in keychain import. The architectural questions remain open.

The project’s own SECURITY.md confirms the most delicate part: credential encryption at rest uses AES-256-GCM with scrypt derivation, and it becomes active once the user sets STORAGE_ENCRYPTION_KEY. Leave that variable empty and the server still starts, with keys sitting in plaintext on the filesystem. The same 20 July review flags a default JWT secret, omniroute-default-secret-change-me, which anyone familiar with the project can use to forge an admin token against a factory-configured instance. The prompt-injection guardrails read the first 16,384 bytes of each request and, when they hit an internal error, let it through.

The weight carried by one person

The README claims “500+ contributors”. Public counters describe a different distribution: 261 recorded contributors, with 2,968 commits out of 3,264 signed by the lead developer and 188 by the second. Release cadence runs at roughly two versions a day, 287 npm publications in 149 days. At that speed regressions arrive: 3.8.48, on 14 July, is a hotfix for 3.8.47, which crashed at startup on missing files, the third episode in the same error class.

The headline compression figure deserves a close reading too. The README states savings of 15-95 % with an average near 89 %; independent measurement reports 18 % on coding work, 50 % on content generation, 65 % on chatbots, averaging 42 %, with 50-200 milliseconds added per request. Compression here is lossy by design: it shrinks the context the model receives, and that effect stays invisible until an answer comes back thinner than expected.

What holds up

Filing OmniRoute away as pure cleverness would be a mistake. Three ideas hold up well. Quota-aware fallback, which treats a plan limit as an ordinary operational event rather than an error to handle by hand. Local-first design, with keys and traffic staying on the machine of the person doing the work, the same direction European sovereign stacks are taking (AI Privacy First starts there). And cost telemetry returned in the headers of every response, which makes consumption readable while it happens rather than at the end of the month.

There is also a market signal here. Twenty-eight thousand stars in a few weeks measure how sharply the cost and fragmentation of model providers is felt. The gateway layer is turning into infrastructure in its own right, with alternatives already mature on different fronts: OpenRouter as a managed service with SOC 2 certification and a 5.5 % fee on credits, LiteLLM as a self-hosted project with per-key budgets, RBAC and audit logs. Teams that need governance already have somewhere to find it.

Three low-cost moves

For anyone running technology inside a company, the practical issue is quiet diffusion: a gateway like this installs in two minutes on a developer’s laptop, reads the system keychain and installs a root certificate, and IT finds out afterwards. Three actions are worth the effort, and they hold whichever tool you pick.

First: write down which model endpoints company devices may reach, and treat root-certificate installation as an event that requires explicit approval. Second: check the terms of service of the providers whose free tiers your company consumes, because contractual risk follows the account, and the account is often a corporate one. Third: ask whoever proposes a gateway the three answers that matter, where credentials end up, what the audit trail produces, and who responds within what time to a security report. A young project can answer well: the point is to have the answer in writing before adoption rather than after the incident.