Back to projects
AI & Automation
Private AI stack — Self-hosted LLM agents on Proxmox
A self-hosted AI infrastructure running local language models and agents on a Proxmox mini-server, connected securely to a GPU workstation.
- 0 €
- API cost per request
- 100%
- Data kept on-premise
Problem
Cloud AI APIs are expensive at volume and send private data to third parties. The goal was an always-on AI agent stack that runs locally and stays private.
Approach
- Proxmox VE on a ThinkCentre mini PC, hosting an Ubuntu VM with Ollama and a quantized open model.
- The agent calls a larger model served by LM Studio on an RTX 3080 workstation, connected over Tailscale.
- Cloudflare Tunnel for safe external access, with monitoring for uptime and logs.
- Iterated from a multi-container design to a simpler single-agent VM that is easier to maintain.
Results
A private, zero-marginal-cost AI backend that powers automations and experiments. [Add specifics.]