Run a local LLM with 42 ham radio tool-calling capabilities in your browser. No cloud, no API keys, no subscriptions.
What you get: A browser UI (Open WebUI) connected to a local LLM (llama.cpp) that can call 42 real-time ham radio tools — live solar conditions, POTA/SOTA spots, WSPR propagation data, IOTA island info, and IONIS propagation analytics.
- NVIDIA GPU with 8+ GB VRAM (16 GB recommended)
- Docker with NVIDIA Container Toolkit
- ~8 GB disk for the default model + ~2 GB for containers
# 1. Clone and configure
git clone https://github.com/qso-graph/llm-stack.git
cd llm-stack
cp .env.example .env # Defaults work for 16 GB VRAM
# 2. Download the LLM model (~5.5 GB)
./scripts/download-model.sh
# 3. Launch
docker compose up -d
# 4. Open browser
# http://localhost:3000Create an account on first visit (local only, not shared anywhere).
The default Docker image supports Turing through Ada Lovelace GPUs. Blackwell GPUs (RTX 5070/5080/5090) need a local build — see Blackwell Build.
| Architecture | GPUs | SM | Default Image | Notes |
|---|---|---|---|---|
| Turing | RTX 2060–2080, T4 | 75 | Yes | |
| Ampere | RTX 3060–3090, A100 | 80/86 | Yes | |
| Ada Lovelace | RTX 4060–4090, L40 | 89 | Yes | |
| Blackwell | RTX 5070–5090, B200 | 100/120 | No | Local build required |
docker build -t ghcr.io/ggml-org/llama.cpp:server-cuda \
-f llm-engine/Dockerfile llm-engine/
docker compose up -dBlackwell NVIDIA driver note: RTX 5080/5090 require the open kernel modules. On RHEL/Rocky/Alma:
sudo dnf module enable nvidia-driver:open-dkms sudo dnf install kmod-nvidia-open-dkmsThe standard
nvidia-driver:latest-dkmswill not work —nvidia-smiwill show "No devices found."
| GPU VRAM | Model | Context | VRAM Used | Notes |
|---|---|---|---|---|
| 8 GB | Qwen2.5-3B Q5_K_M | 8K | ~3 GB | Basic tool calling |
| 16 GB | Qwen2.5-7B Q5_K_M (default) | 16K | ~6.4 GB | Good tool calling, 138 tok/s on RTX 5080 |
| 24 GB | Qwen2.5-14B Q5_K_M | 16K | ~12 GB | Better reasoning |
| 48+ GB | Qwen2.5-32B Q5_K_M | 32K | ~24 GB | Best quality, set LLM_CTX_SIZE=32768 |
To use a different model, download the GGUF file into models/ and update LLM_MODEL in .env.
After launching, connect the MCP tool servers in Open WebUI:
- Admin Panel → Settings → Tools (or Connections → Tool Servers)
- Add each server as type OpenAPI (NOT "MCP Streamable HTTP"):
| Name | URL | Tools |
|---|---|---|
| Solar MCP | http://mcp-tools:8001 |
6 — conditions, alerts, forecast, X-ray, solar wind, band outlook |
| POTA MCP | http://mcp-tools:8002 |
7 — spots, park info, stats, user stats, scheduled, location parks, nearby |
| WSPR MCP | http://mcp-tools:8003 |
8 — spots, band activity, propagation, grid activity, SNR trends |
| SOTA MCP | http://mcp-tools:8004 |
4 — spots, alerts, summit info, nearby summits |
| IOTA MCP | http://mcp-tools:8005 |
6 — island lookup, search, DXCC mapping, nearby groups |
| IONIS MCP | http://mcp-tools:8006 |
11 — propagation analytics (requires datasets, see below) |
- Enable tools per chat — click the wrench icon in the chat input box
- Model settings — in Advanced Params, set Function Calling to Native
Important: Use OpenAPI type, not "MCP Streamable HTTP". Open WebUI's native MCP support is broken as of v0.7.2.
Ask questions like:
- "What are current solar conditions?"
- "Show me live POTA activations in a table"
- "What WSPR propagation is there on 20m right now?"
- "Find SOTA summits near Denver"
- "Look up IOTA group OC-001"
Tip: 7B models sometimes answer from memory instead of calling tools. Prefix your question with the tool name: "Use solar-mcp — what are current conditions?" Larger models (14B+) are better at deciding when to use tools on their own.
graph TD
subgraph net ["Docker: ai-net network"]
LLM["llm-engine :8000\n(GPU)"]
WebUI["open-webui :3000\n(browser)"]
subgraph tools ["mcp-tools container"]
S1["solar-mcp :8001 — 6 tools"]
S2["pota-mcp :8002 — 7 tools"]
S3["wspr-mcp :8003 — 8 tools"]
S4["sota-mcp :8004 — 4 tools"]
S5["iota-mcp :8005 — 6 tools"]
S6["ionis-mcp :8006 — 11 tools"]
end
WebUI -- inference --> LLM
WebUI -- "OpenAPI (mcpo)" --> tools
end
mcpo bridges stdio-based MCP servers into OpenAPI endpoints that Open WebUI consumes as external tools.
IONIS provides 175M+ propagation signatures from WSPR, RBN, Contest, and PSK Reporter data. To enable the 11 IONIS tools:
- Download datasets from SourceForge (~15 GB)
- Set
IONIS_DATA_DIRin.envto the download directory - Launch with the IONIS override:
docker compose -f docker-compose.yaml -f docker-compose.ionis.yaml up -dWithout IONIS datasets, the other 31 tools (solar, POTA, WSPR, SOTA, IOTA) still work.
To expose your instance publicly:
- Create a tunnel at Cloudflare Zero Trust
- Set
CLOUDFLARE_TUNNEL_TOKENin.env - Launch with the tunnel profile:
docker compose --profile tunnel up -d# Pull latest MCP server versions from PyPI
docker compose build --no-cache mcp-tools
docker compose up -d mcp-toolsGPU not detected in container:
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
docker run --rm --gpus all nvidia/cuda:12.8.1-base-ubuntu22.04 nvidia-smiTools not calling: Enable tools via the wrench icon in the chat input. Set Function Calling to Native in model Advanced Params.
Connection refused on tool servers: Verify mcp-tools is on the same Docker network: docker network inspect llm-stack_ai-net
Out of VRAM: Reduce LLM_CTX_SIZE in .env (try 8192) or use a smaller quantization (Q4_K_M).
Model not loading: Verify the GGUF file exists in models/ and the filename matches LLM_MODEL in .env.
Blackwell GPU — "No devices found": Switch to open kernel modules. See Blackwell Build.
| Port | Service | Purpose |
|---|---|---|
| 3000 | Open WebUI | Browser chat UI |
| 8000 | llm-engine | LLM inference API (GPU) |
| 8001–8006 | mcpo | MCP tool servers (OpenAPI proxy) |
MIT
- Full documentation — setup guide, GPU compatibility, tool reference
- qso-graph MCP servers — 13 ham radio MCP packages on PyPI
- IONIS-AI — propagation prediction datasets
- Open WebUI — browser interface
- llama.cpp — LLM inference engine
- mcpo — MCP-to-OpenAPI proxy