RouterLM · open design, MIT
AI that keeps working when a provider doesn't.
RouterLM is one small service in front of every model your office uses. When a provider goes down, rate-limits you, changes its terms or retires the model you built on, the work moves to the next model on the list. The last model on every list runs on a box in your building, so work continues even with the internet down.
What it does
- One address for every program. Programs call RouterLM instead of a vendor. Pointing a program at it means changing one setting.
- Jobs, not model names. Programs ask for "answer" or "summarize." A list a person wrote decides which model does it, best first.
- Plain rules for moving on. A model that's down, too slow, rate-limited or out of credit hands off to the next one.
- A local model ends every list. When the internet or every provider is down, work keeps going on the box on the shelf.
- Keeping data in is a switch. Mark a program or a job private, and its requests never leave the building, even when a model fails.
- Every answer labeled with the model that wrote it, and a status light that says when something's wrong.
With Capsule Base
Point Capsule Base at RouterLM and it keeps answering when a provider stops. Mark it private and your questions never leave the building: the records, the index and the model that writes the answer all sit on the same shelf.
Open, and built per business
The design is published under MIT: the architecture, sizing the box, what it costs to keep working, and the appliance itself. Builds are custom to each business: its records, its hardware, its rules about what may leave the building.
The longer argument for owning this layer is in The Last Interface.