You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I searched existing issues before opening this one
Expected behavior
Docker should let me upgrade from devicemapper to overlay2 storage engine.
Actual behavior
Get error when starting Docker -
failed to mount overlay: invalid argument storage-driver=overlay2
driver not supported
It's my understanding that Docker uses overlay2 by default, but I seem to be stuck with devicemapper. I was having problems with that eating up disk space, so wanted to switch to overlay2.
Steps to reproduce the behavior
$ sudo systemctl stop docker
$ sudo rm -rf /var/lib/docker
$ sudo nano /etc/docker/daemon.json
$ sudo cat /etc/docker/daemon.json
{
"storage-driver": "overlay2"
}
$ sudo systemctl start docker
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
$ sudo dockerd --debug
INFO[2022-05-25T06:48:33.025968569Z] Starting up
DEBU[2022-05-25T06:48:33.027335891Z] Listener created for HTTP on unix (/var/run/docker.sock)
INFO[2022-05-25T06:48:33.028909628Z] detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf
DEBU[2022-05-25T06:48:33.031405107Z] Golang's threads limit set to 27360
INFO[2022-05-25T06:48:33.034216750Z] parsed scheme: "unix" module=grpc
INFO[2022-05-25T06:48:33.034358207Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2022-05-25T06:48:33.034593580Z] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2022-05-25T06:48:33.035834028Z] ClientConn switching balancer to "pick_first" module=grpc
DEBU[2022-05-25T06:48:33.034378623Z] metrics API listening on /var/run/docker/metrics.sock
INFO[2022-05-25T06:48:33.046671395Z] parsed scheme: "unix" module=grpc
INFO[2022-05-25T06:48:33.046786019Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2022-05-25T06:48:33.046875852Z] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2022-05-25T06:48:33.046921352Z] ClientConn switching balancer to "pick_first" module=grpc
DEBU[2022-05-25T06:48:33.049799494Z] Using default logging driver json-file
DEBU[2022-05-25T06:48:33.049948826Z] [graphdriver] trying provided driver: overlay2
DEBU[2022-05-25T06:48:33.049809202Z] processing event stream module=libcontainerd namespace=plugins.moby
ERRO[2022-05-25T06:48:33.054543121Z] failed to mount overlay: invalid argument storage-driver=overlay2
DEBU[2022-05-25T06:48:33.055641820Z] Cleaning up old mountid : start.
failed to start daemon: error initializing graphdriver: driver not supported
Output of docker version:
Client: Docker Engine - Community
Version: 20.10.12
API version: 1.41
Go version: go1.16.12
Git commit: e91ed57
Built: Mon Dec 13 11:44:28 2021
OS/Arch: linux/arm64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.12
API version: 1.41 (minimum version 1.12)
Go version: go1.16.12
Git commit: 459d0df
Built: Mon Dec 13 11:43:05 2021
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.4.12
GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.7.1-docker)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 20.10.12
Storage Driver: devicemapper
Pool Name: docker-0:28-398286-pool
Pool Blocksize: 65.54kB
Base Device Size: 10.74GB
Backing Filesystem: ext4
Udev Sync Supported: true
Data file: /dev/loop0
Metadata file: /dev/loop1
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Data Space Used: 240.1MB
Data Space Total: 107.4GB
Data Space Available: 23.87GB
Metadata Space Used: 17.47MB
Metadata Space Total: 2.147GB
Metadata Space Available: 2.13GB
Thin Pool Minimum Free Space: 10.74GB
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
Library Version: 1.02.167 (2019-11-30)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc version: v1.0.2-0-g52b36a2
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.60
Operating System: Ubuntu 20.04.3 LTS
OSType: linux
Architecture: aarch64
CPUs: 6
Total Memory: 3.745GiB
Name: l99-682719a73ffd
ID: ZHEX:YJ4R:I4A4:TV4G:U6HO:5FCI:CXUA:QGQ5:BK57:LNMF:ZNB3:234Y
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release.
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
The error might happen when the root file system (/) is itself an overlay. You can check it using df -h / and looking for the file system column value.
Based on this, [o]verlay on top of overlay is not supported by the kernel, which would be the case when the root FS is an overlay and you want to run Docker in an overlay on top of it.
One can work this around using additional partition (formatted to, e.g., ext4) mounted at /var/lib/docker (or /var/lib/docker/overlay2). Don’t forget to stop Docker service, copy the data from /var/lib/docker somewhere, mount the partition, copy the data back and update /etc/fstab to make sure the partition is mounted at system startup.
Did you by any chance enable the casefold feature in the ext4 partition that holds /var/lib/docker? This is a relatively new feature that allows specific directories to be made case-insensitive through a file/directory attribute. I'm not sure how many distros already enable the casefold feature in ext4 partitions by default in new installs.
Expected behavior
Docker should let me upgrade from devicemapper to overlay2 storage engine.
Actual behavior
Get error when starting Docker -
It's my understanding that Docker uses overlay2 by default, but I seem to be stuck with devicemapper. I was having problems with that eating up disk space, so wanted to switch to overlay2.
Steps to reproduce the behavior
Output of
docker version
:Output of
docker info
:Additional environment details (AWS, VirtualBox, physical, etc.)
The text was updated successfully, but these errors were encountered: