Local Models with Ollama
Run open models (Llama, Qwen, Hermes) on your own machine.
Ollama runs open-source models locally β no cloud, no API key, no data leaving your machine.
Install & run
# macOS / Linux / Windows (WSL2)
curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1 # chat with Llama
ollama pull qwen2.5-coder # a strong open coding model
Ollama exposes a local API (http://localhost:11434) that any tool can use.
Use it with your agents
# Aider with a local model
aider --model ollama/llama3.1
# Codex CLI can also point at compatible endpoints
codex --config '{"model_provider":"ollama","model":"qwen2.5-coder"}'
Choosing a model
| Model | Strengths | | --- | --- | | llama3.1 / llama3.3 | Solid generalist, fast | | qwen2.5-coder | Strong at code for its size | | deepseek-coder | Excellent code, chatty | | hermes3 | Built for tool use / function calling | | codestral | Mistral's coding specialist |
Trade-offs
- Pro: privacy, free, offline, no rate limits.
- Con: smaller context, slower, and weaker on the hardest multi-file tasks than frontier cloud models.
A common pro workflow: use a local model for autocomplete and simple refactors, and a cloud model for big agent tasks. Best of both.