I run a small server with Proxmox, and I’m wondering what are your opinions on running Docker in separate LXC containers vs. running a specific VM for all Docker containers?
I started with LXC containers because I was more familiar with installing services the classic Linux way. I later added a VM specifically for running Docker containers. I’m thinking if I should continue this strategy and just add some more resources to the docker VM.
On one hand, backups seem to be easier with individual LXCs (I’ve had situations where I tried to update a Docker container but the new container broke the existing configuration and found it easiest just to restore the entire VM from backup). On the otherhand, it seems like more overhead to install Docker in each individual LXC.
What’s the purpose of running container in a container? Why not install docker on your host machine?
If you do that, Docker is stuck on that host. If it’s in an LXC it can move to another host. Plus, backing up and snapshotting are easier IMO.
Snapshotting in docker is as easy as
docker commit. After that you can back it up withdocker save. Then move to another host, but not without downtime.However normally you need to backup/move only volumes attached to containers. If that’s not the way how you like to organize your services, you likely don’t need docker.
Dockers ‘take-over-system’ style of network management will interfere with proxmox networking.
Regardless of VM or LXC, I would only install docker once. There’s generally no need to create multiple docker VMs/LXCs on the same host. Unless you have a specific reason; like isolating outside traffic by creating a docker setup for only public services.
Backups are the same with VM or LXC on Proxmox.
The main advantages of LXC that I can think of:
- Slightly less resource overhead, but not much (debian minimal or alpine VM is pretty lightweight already).
- Ability to pass-through directories from the host.
- Ability to pass-through hardware acceleration from a GPU, without passing through the entire GPU.
- Ability to change CPU cores or RAM while it’s running.
I use individual lxc for each docker compose so I don’t have to revert 8 services at once if I need to restore.
I would also argue that an alpine lxc runs in 22mb ram by itself … Significantly smaller footprint on disk and in memory. But most importantly, lxc can actually share memory space effectively, one doesn’t need to reserve blocks of ram.
