|
|
@@ -0,0 +1,126 @@
|
|
|
+---
|
|
|
+title: Docker
|
|
|
+breadcrumbs:
|
|
|
+- title: Configuration
|
|
|
+- title: Virtualization & Containerization
|
|
|
+---
|
|
|
+{% include header.md %}
|
|
|
+
|
|
|
+Using **Debian**, unless otherwise stated.
|
|
|
+
|
|
|
+## Setup
|
|
|
+
|
|
|
+1. Install: [Install Docker Engine on Debian (Docker Documentation)](https://docs.docker.com/engine/install/debian/).
|
|
|
+1. (Optional) Setup swap limit:
|
|
|
+ - If `docker info` contains `WARNING: No swap limit support`, it's not working properly and should maybe be fixed.
|
|
|
+ - Enabling/fixing it incurs a small performance degredation and is optional but recommended.
|
|
|
+ - In `/etc/default/grub`, add `cgroup_enable=memory swapaccount=1` to `GRUB_CMDLINE_LINUX`.
|
|
|
+ - Run `update-grub` and reboot the system.
|
|
|
+1. Configure `/etc/docker/daemon.json`:
|
|
|
+ - Enable IPv6: `"ipv6": true` and `"fixed-cidr-v6": "<ipv6-subnet>/64"`
|
|
|
+ - Note that IPv6 it not NATed like IPv4 is in Docker.
|
|
|
+ - Use something like [Simple DNS Plus' Private IPv6 address range](https://simpledns.plus/private-ipv6) to generate an IPv6 ULA prefix.
|
|
|
+ - Set DNS servers for containers: `"dns": ["1.1.1.1", "2606:4700:4700::1111"]`
|
|
|
+ - If not set, containers will use `8.8.8.8` and `8.8.4.4` by default.
|
|
|
+ - `/etc/resolv.conf` is limited to only three name servers, so don't provide too many. One may be set by the container itself.
|
|
|
+ - (Optional) Disable automatic IPTables rules: `"iptables": false`
|
|
|
+1. Setup IPv6 firewall and NAT:
|
|
|
+ - By default, Docker does not add any IPTables NAT rules or filter rules, which leaves Docker IPv6 networks open (bad) and requires using a routed prefix (sometimes bad).
|
|
|
+ - For the changes below, open `/etc/docker/daemon.json`.
|
|
|
+ - Set `"ipv6": true` to enable IPv6 support at all.
|
|
|
+ - Set `"ip6tables": true` to enable adding filter and NAT rules to IP6Tables (required for both security and NAT).
|
|
|
+ - Set `"fixed-cidr-v6"` to some IPv6 prefix shorter than 64 bits, for use by Docker networks. If not using IPv6 NAT, this needs to be routable.
|
|
|
+1. (Optional) Change default DNS servers for containers:
|
|
|
+ - In `/etc/docker/daemon.json`, set `"dns": ["1.1.1.1", "2606:4700:4700::1111"]` (example using Cloudflare) (3 servers max).
|
|
|
+ - It defaults to `8.8.8.8` and `8.8.4.4` (Google).
|
|
|
+1. (Optional) Enable Prometheus metrics endpoint:
|
|
|
+ - This only exports internal Docker metrics, not anything about the containers (use cAdvisor for that).
|
|
|
+ - In `/etc/docker/daemon.json`, set `"experimental": true` and `"metrics-addr": "[::]:9323"`.
|
|
|
+1. (Optional) Allow non-root users to use Docker:
|
|
|
+ - Add them to the `docker` group.
|
|
|
+ - This is not recommended as it effectively grants them root access, so sudo is a cleaner alternative.
|
|
|
+
|
|
|
+## Usage
|
|
|
+
|
|
|
+**TODO** Clean up this section.
|
|
|
+
|
|
|
+- Miscellanea:
|
|
|
+ - Show disk usage: `docker system df -v`
|
|
|
+- Cleanup:
|
|
|
+ - Prune unused images: `docker image prune -a`
|
|
|
+ - Prune unused volumes: `docker volume prune`
|
|
|
+- Docker run options:
|
|
|
+ - Set name: `--name=<name>`
|
|
|
+ - Run in detatched mode: `-d`
|
|
|
+ - Run using interactive terminal: `-it`
|
|
|
+ - Automatically remove when stopped: `--rm`
|
|
|
+ - Automatically restart: `--restart=unless-stopped`
|
|
|
+ - Use "tini" as entrypoint and use PID 1: `--init`
|
|
|
+ - Set env var: `-e <var>=<val>`
|
|
|
+ - Publish network port: `-p <host-port>:<cont-port>[/udp]`
|
|
|
+ - Mount volume: `-v <vol>:<cont-path>` (`<vol>` must have a path prefix like `./` or `/` if it is a directory and not a named volume)
|
|
|
+
|
|
|
+### Networking
|
|
|
+
|
|
|
+**TODO** Clean up this subsection too.
|
|
|
+
|
|
|
+- Containers in production should not use the default Docker networks.
|
|
|
+- Try to isolate container communication into as small networks as possible (e.g. one network per group of containers for an application).
|
|
|
+- Docker doesn't integrate with ip6tables at all, meaning certain IPv6 features are lacking. For instance, IPv6 is not NATed like IPv4 and ICC can't be disabled. NAT66 shouldn't generally be used in the first place, but the lack of it means IPv6 requires a bit of extra configuration to get it working with containers. IPv6 routing and port publishing work as they should, though, as they don't use ip6tables.
|
|
|
+- Network types:
|
|
|
+ - Bridge: A plain bridge where all containers and the host can communicate. Can optionally be directly connected to a host bridge, but that doesn't always work as expected. Vulnerable to ARP/NDP spoofing.
|
|
|
+ - Overlay: Overlay network for swarm stuff.
|
|
|
+ - Host: The container use the network stack of the host. Ports are published directly to the host.
|
|
|
+ - MACVLAN: Bridges connected to a host (parent) interface, allowing containers to be connected to a network the host is part of. Can optionally use trunking on the host interface. All communication between containers and the host is dropped (consider using a host-connected bridge if you need this).
|
|
|
+ - L2 IPVLAN: Similar to MACVLAN, but all containers use the host's MAC address. Containers can communicate, but the host can't communicate with any containers.
|
|
|
+ - L3 IPVLAN: Every VM uses a separate subnet and all communication, internally and externally, is routed. Should avoid ARP/NDP spoofing. (**TODO:** Containers and the host can communicate?)
|
|
|
+- Create:
|
|
|
+ - Create bridged network: `docker network create --driver=bridge --subnet=<ipv4-net> --ipv6 --subnet=<ipv6-net> <name>`
|
|
|
+ - Create external bridged network (experimental, doesn't work as intented in some scenarios): `docker network create --driver=bridge --subnet=<ipv4-net> --gateway=<ipv4-gateway> --ipv6 --subnet=<ipv6-net> --gateway=<ipv6-gateway> -o "com.docker.network.bridge.name=<host-if> <name>`
|
|
|
+ - Create MACVLAN: `docker network create --driver=macvlan --subnet=<ipv4-net> --gateway=<ipv4-gateway> --ipv6 --subnet=<ipv6-net> --gateway=<ipv6-gateway> -o parent=<netif>[.<vid>] <name>`
|
|
|
+ - Create L2 IPVLAN with parent interface: `docker network create --driver=ipvlan --subnet=<ipv4-net> --gateway=<ipv4-gateway> --ipv6 --subnet=<ipv6-net> --gateway=<ipv6-gateway> -o parent=<netif> <name>`
|
|
|
+- Use:
|
|
|
+ - Run container with network: `docker run --network=<net-name> --ip=<ipv4-addr> --ip6=<ipv6-addr> --dns=<dns-server> [...] <image>`
|
|
|
+- Disable IPv4 and IPv6 NAT/masquerade for a bridge network: `docker network create <...> -o "com.docker.network.bridge.enable_ip_masquerade=false" <name>`
|
|
|
+- Set the Linux name of a bridge network: `docker network create <...> -o "com.docker.network.bridge.name=<name>" <name>`
|
|
|
+
|
|
|
+## Miscellanea
|
|
|
+
|
|
|
+- Use [watchtower](https://github.com/containrrr/watchtower) to automatically update images and restart containers.
|
|
|
+- Use [cAdvisor](https://github.com/google/cadvisor) to monitor containers (including a Prometheus metrics endpoint).
|
|
|
+
|
|
|
+### IPv6 Support
|
|
|
+
|
|
|
+- TL;DR: Docker doesn't prioritize implementing IPv6 properly.
|
|
|
+- While IPv4 uses IPTables filter rules for firewalling and and IPTables NAT rules for masquerading and port forwarding, it generally uses no such mechanisms when enabling IPv6 (using `"ipv6": true`). Setting `"ip6tables": true` explicitly (it's disabled by default) is required to mimic the IPv4 behavior of filtering and NAT-ing. To disable NAT masquerading for both IPv4 and IPv6, set `enable_ip_masquerade=false` on individual networks. Disabling NAT masquerading for only IPv6 is not yet possible. (See [moby/moby #13481](https://github.com/moby/moby/issues/13481), [moby/moby #21951](https://github.com/moby/moby/issues/21951), [moby/moby #25407](https://github.com/moby/moby/issues/25407), [moby/libnetwork #2557](https://github.com/moby/libnetwork/issues/2557).)
|
|
|
+- IPv6-only networks (without IPv4) are not supported. (See [moby/moby #32675](https://github.com/moby/moby/issues/32675), [moby/libnetwork #826](https://github.com/moby/libnetwork/pull/826).)
|
|
|
+- The userland proxy (enabled by default, can be disabled) accepts both IPv4 and IPv6 incoming traffic but uses only IPv4 toward containers, which replaces the IPv6 source address with an internal IPv4 address (I'm not sure which), effectively hiding the real address and may bypass certain defences as it's apparently coming from within the local network. It also has other non-IPv6-related problems. (See [moby/moby #11185](https://github.com/moby/moby/issues/11185), [moby/moby #14856](https://github.com/moby/moby/issues/14856), [moby/moby #17666](https://github.com/moby/moby/issues/17666).)
|
|
|
+
|
|
|
+## Docker Compose
|
|
|
+
|
|
|
+### Setup
|
|
|
+
|
|
|
+1. Install Docker: See above.
|
|
|
+1. Install: [Docker Documentation: Install Docker Compose](https://docs.docker.com/compose/install/).
|
|
|
+1. Install command completion: [Docker Documentation: Command-line completion](https://docs.docker.com/compose/completion/).
|
|
|
+
|
|
|
+### Troubleshooting
|
|
|
+
|
|
|
+#### Fix Docker Compose No-Exec Tmp-Dir
|
|
|
+
|
|
|
+Docker Compose will fail to work if `/tmp` has `noexec`.
|
|
|
+
|
|
|
+1. Move `/usr/local/bin/docker-compose` to `/usr/local/bin/docker-compose-normal`.
|
|
|
+1. Create `/usr/local/bin/docker-compose` with the contents below and make it executable.
|
|
|
+1. Create the new TMPDIR dir.
|
|
|
+
|
|
|
+New `docker-compose`:
|
|
|
+
|
|
|
+```sh
|
|
|
+#!/bin/bash
|
|
|
+# Some dir without noexec
|
|
|
+export TMPDIR=/var/lib/docker-compose-tmp
|
|
|
+/usr/local/bin/docker-compose-normal "$@"
|
|
|
+```
|
|
|
+
|
|
|
+{% include footer.md %}
|
Håvard O. Nordstrand