Blame
|
1 | # Docker Swarm |
||||||
| 2 | ||||||||
| 3 | ← [[Home]] |
|||||||
| 4 | ||||||||
|
5 | Five-node Swarm cluster managed from PCT 108 (network, 192.168.2.82). PCT 109 (ai) is **not** a Swarm node — all its services are standalone Docker Compose. |
||||||
|
6 | |||||||
| 7 | | Node | Hostname | IP | Role | |
|||||||
| 8 | |------|----------|----|------| |
|||||||
| 9 | | PCT 101 | downloads | 192.168.2.190 | Worker | |
|||||||
| 10 | | PCT 102 | media-core | 192.168.2.191 | Worker | |
|||||||
| 11 | | PCT 104 | documents | 192.168.2.105 | Worker | |
|||||||
| 12 | | PCT 107 | debian | 192.168.2.81 | Worker | |
|||||||
| 13 | | PCT 108 | network | 192.168.2.82 | **Manager** | |
|||||||
| 14 | ||||||||
| 15 | --- |
|||||||
| 16 | ||||||||
| 17 | ## Key Commands |
|||||||
| 18 | ||||||||
| 19 | All Swarm commands run from the Proxmox host via `pct exec 108 --`. |
|||||||
| 20 | ||||||||
| 21 | ```bash |
|||||||
| 22 | # Status |
|||||||
| 23 | pct exec 108 -- docker node ls |
|||||||
| 24 | pct exec 108 -- docker stack ls |
|||||||
| 25 | pct exec 108 -- docker service ls |
|||||||
| 26 | ||||||||
| 27 | # Inspect a service |
|||||||
|
28 | pct exec 108 -- docker service ps <service> # failed replica history |
||||||
|
29 | pct exec 108 -- docker service logs <service> --tail 50 |
||||||
| 30 | ||||||||
|
31 | # Deploy / redeploy |
||||||
|
32 | pct exec 108 -- docker stack deploy -c <compose> <stack> |
||||||
|
33 | pct exec 108 -- docker service update --force <service> # restart + fresh overlay IP |
||||||
|
34 | |||||||
| 35 | # Check containers on a specific node |
|||||||
| 36 | pct exec 102 -- docker ps -a --filter name=<name> |
|||||||
|
37 | pct exec 104 -- docker ps |
||||||
|
38 | ``` |
||||||
| 39 | ||||||||
| 40 | --- |
|||||||
| 41 | ||||||||
| 42 | ## Compose Files |
|||||||
| 43 | ||||||||
| 44 | Stored in Portainer on PCT 108: |
|||||||
| 45 | ``` |
|||||||
| 46 | /mnt/tank/appdata/portainer/compose/<id>/docker-compose.yml |
|||||||
| 47 | ``` |
|||||||
| 48 | ||||||||
|
49 | **Portainer env var gotcha:** Portainer stores stack env vars in its internal DB (`portainer.db`) — there are no `.env` files on disk. When redeploying via CLI, export vars manually: |
||||||
|
50 | ```bash |
||||||
| 51 | pct exec 108 -- bash -c 'export VAR=val && docker stack deploy -c <compose> <stack>' |
|||||||
| 52 | ``` |
|||||||
| 53 | ||||||||
|
54 | ### Portainer Directory → Stack Mapping |
||||||
|
55 | |||||||
| 56 | | Dir ID | Stack | |
|||||||
| 57 | |--------|-------| |
|||||||
| 58 | | 2 | documents-linkwarden | |
|||||||
| 59 | | 4 | documents-paperless | |
|||||||
| 60 | | 6 | network-guacamole | |
|||||||
| 61 | | 7 | documents-nextcloud | |
|||||||
| 62 | | 10 | documents-immich | |
|||||||
| 63 | | 22 | documents-homarr | |
|||||||
| 64 | | 23 | authentik | |
|||||||
| 65 | | 25 | dozzel | |
|||||||
| 66 | | 26 | media-core | |
|||||||
| 67 | | 28 | documents-actual-budget | |
|||||||
| 68 | | 31 | network-cloudbeaver | |
|||||||
| 69 | | 32 | romm | |
|||||||
| 70 | ||||||||
| 71 | --- |
|||||||
| 72 | ||||||||
| 73 | ## Swarm Labels vs Container Labels |
|||||||
| 74 | ||||||||
| 75 | In Docker Swarm, **only `deploy.labels` are read by Traefik** — container-level `labels:` are ignored. |
|||||||
| 76 | ||||||||
| 77 | ```yaml |
|||||||
| 78 | services: |
|||||||
| 79 | myservice: |
|||||||
| 80 | deploy: |
|||||||
| 81 | labels: |
|||||||
| 82 | - "traefik.enable=true" |
|||||||
| 83 | - "traefik.http.routers.myservice.rule=Host(`myservice.carr-family.org`)" |
|||||||
|
84 | - "traefik.http.services.myservice.loadbalancer.server.port=8080" |
||||||
|
85 | ``` |
||||||
| 86 | ||||||||
| 87 | --- |
|||||||
| 88 | ||||||||
|
89 | ## Cross-Provider Middleware Reference |
||||||
| 90 | ||||||||
| 91 | Middlewares defined in `routes.yml` (file provider) must be referenced with `@file` suffix in Swarm labels: |
|||||||
| 92 | ```yaml |
|||||||
| 93 | - "traefik.http.routers.myservice.middlewares=lan-only@file" |
|||||||
| 94 | ``` |
|||||||
| 95 | Plain names default to `@swarm` and return 404. |
|||||||
| 96 | ||||||||
| 97 | --- |
|||||||
| 98 | ||||||||
|
99 | ## Overlay Network Health Check |
||||||
| 100 | ||||||||
|
101 | Services on PCT 102 can develop stale VXLAN attachments after redeployments, causing 504s even though the service shows `1/1` and the container is healthy. |
||||||
|
102 | |||||||
|
103 | **Diagnose:** |
||||||
|
104 | ```bash |
||||||
| 105 | TRAEFIK=$(pct exec 108 -- docker ps -q --filter name=traefik) |
|||||||
| 106 | IP=$(pct exec 102 -- docker inspect <container-name> --format '{{(index .NetworkSettings.Networks "traefik-public").IPAddress}}') |
|||||||
| 107 | pct exec 108 -- docker exec $TRAEFIK ping -c 2 $IP |
|||||||
|
108 | ``` |
||||||
| 109 | ||||||||
| 110 | **Fix:** |
|||||||
| 111 | ```bash |
|||||||
|
112 | pct exec 108 -- docker service update --force <service-name> |
||||||
| 113 | ``` |
|||||||
| 114 | ||||||||
|
115 | All 11 media-core services were force-updated 2026-05-13 to clear a full-stack stale attachment event. |
||||||
| 116 | ||||||||
|
117 | --- |
||||||
| 118 | ||||||||
|
119 | ## Standalone Containers (not Swarm-managed) |
||||||
| 120 | ||||||||
| 121 | Several services run as `docker compose` or `docker run` directly on specific nodes, outside Swarm. They need static routes in Traefik's `routes.yml` since Swarm label discovery doesn't apply. |
|||||||
| 122 | ||||||||
| 123 | | Container | Node | IP:Port | Route | |
|||||||
| 124 | |-----------|------|---------|-------| |
|||||||
| 125 | | `qbittorrent-mam` | PCT 101 | 192.168.2.190:8080 | static route | |
|||||||
| 126 | | `qbittorrent-vpn` | PCT 101 | 192.168.2.190:8081 | static route | |
|||||||
| 127 | | `gluetun-proton` | PCT 101 | — | VPN gateway only | |
|||||||
| 128 | | `qui` | PCT 101 | 192.168.2.190:7476 | static route | |
|||||||
| 129 | | `jellyfin` | PCT 101 | 192.168.2.191:8096 | static route | |
|||||||
| 130 | | `otterwiki` | PCT 104 | 192.168.2.105:8081 | static route | |
|||||||
| 131 | | `litellm` + `postgres` | PCT 109 | 192.168.2.83:4000 | static route | |
|||||||
| 132 | | `n8n` + `qdrant` | PCT 109 | 192.168.2.83:5678 | static route | |
|||||||
| 133 | | `openclaw` | PCT 109 | 192.168.2.83:18789 | static route | |
|||||||
| 134 | | `odysseus` | PCT 109 | 192.168.2.83:7000 | static route | |
|||||||
| 135 | | `firecrawl` | PCT 109 | 192.168.2.83:3002 | no route | |
|||||||
| 136 | | `whisper` | PCT 109 | 192.168.2.83:8001 | no route | |
|||||||
| 137 | | `proton-bridge` | PCT 109 | 192.168.2.83:1025/1143 | no route | |
|||||||
| 138 | | Pterodactyl Panel | PCT 300 | 192.168.2.136:80 | static route (lan-only) | |
|||||||
|
139 | |||||||
|
140 | --- |
||||||
|
141 | |||||||
|
142 | ## General Compose Notes |
||||||
| 143 | ||||||||
| 144 | - All services use `PUID=1000 PGID=1000` and `TZ=America/Toronto` unless noted |
|||||||
| 145 | - `gai_conf_ipv4` is an external Docker config used by Prowlarr to force IPv4 DNS |
|||||||
| 146 | - Before adding bind mounts: `mkdir -p /tank/appdata/<app>/<dir> && chown 1000:1000 ...` on the host first |
|||||||
| 147 | - PCT 108 cannot read `/tank/appdata/<app>/` files directly from other nodes' compose dirs — always pipe compose files in via stdin when deploying from a compose file that lives on another node |
|||||||
