Many businesses want an AI assistant but are uneasy about sending their data to a third party. I built a local-first assistant to show it can be done: it runs on local models, orchestrated with LangGraph, and uses a suite of 17 tools exposed over MCP, the Model Context Protocol.
Here is how the pieces fit together, and why each choice was made.
Why local-first
With local models, sensitive documents and conversations stay on your own machines. It also means the assistant keeps working without an internet connection, and the running costs do not grow with every question asked.
The three parts
- llama.cpp runs the models. I can swap one model for another on the fly, using a small model for quick tasks and a larger one when the work needs it.
- LangGraph orchestrates. It decides which step comes next, which tool to call, and when to stop and check.
- MCP provides the tools. Seventeen tools sit behind one standard interface, so the assistant can use them without custom wiring for each.
The model is the easy part. The tools, the rules and the checkpoints are what make an assistant useful.
Why MCP matters
Before MCP, every tool needed its own connection code. A shared standard means a tool written once can be used by any compatible assistant, and new tools can be added without rebuilding the rest.
What I would tell anyone building one
- Start with a few tools that matter and add more only when they are needed.
- Keep a person in the loop for anything with consequences.
- Choose model size by task. Bigger is not always better for routine work.
If you are weighing up a private, local AI assistant for your own team, I am happy to talk it through.