CentOS Linux release 7.9.2009 (Core) docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't get final child's PID from pipe: EOF: unknown

[mrxlhm]

env

OS

uname -a

Linux VM-0-4-centos 3.10.0-1160.108.1.el7.x86_64 #1 SMP Thu Jan 25 16:17:31 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 

cat /etc/centos-release

CentOS Linux release 7.9.2009 (Core)

docker daemon.json

{
  "userns-remap":"default"
}

docker info

Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.6
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
Server:
 Containers: 16
  Running: 0
  Paused: 0
  Stopped: 16
 Images: 2
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  userns
 Kernel Version: 3.10.0-1160.108.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.545GiB
 Name: VM-0-4-centos
 ID: a6ec06bf-07c3-4fc8-8ad8-f9e41309b248
 Docker Root Dir: /var/lib/docker/100000.100000
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

error

docker run hello-world
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't get final child's PID from pipe: EOF: unknown

log

docker.service

journalctl -fu docker.service

Mar 12 14:31:28 VM-0-4-centos dockerd[25629]: time="2024-03-12T14:31:28.864364787+08:00" level=error msg="stream copy error: reading from a closed fifo"
Mar 12 14:31:28 VM-0-4-centos dockerd[25629]: time="2024-03-12T14:31:28.864416167+08:00" level=error msg="stream copy error: reading from a closed fifo"
Mar 12 14:31:28 VM-0-4-centos dockerd[25629]: time="2024-03-12T14:31:28.905311955+08:00" level=error msg="Handler for POST /v1.44/containers/f84443817d1600bbb96dd4ad46056ebd0a04d6cf44cde5fb9ab7c6800480533d/start returned error: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't get final child's PID from pipe: EOF: unknown"

containerd.service

Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.062332781+08:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.062453929+08:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.062479668+08:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.062657805+08:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.runtime.v2.task/moby/22fd6cedcf87252ef2585db60da56d1692f7aa64d4d3f7d77df51f1342931833 pid=5807 runtime=io.containerd.runc.v2
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.082531736+08:00" level=info msg="shim disconnected" id=22fd6cedcf87252ef2585db60da56d1692f7aa64d4d3f7d77df51f1342931833
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.082584785+08:00" level=warning msg="cleaning up after shim disconnected" id=22fd6cedcf87252ef2585db60da56d1692f7aa64d4d3f7d77df51f1342931833 namespace=moby
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.082595325+08:00" level=info msg="cleaning up dead shim"
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.088379390+08:00" level=warning msg="cleanup warnings time=\"2024-03-12T14:32:51+08:00\" level=info msg=\"starting signal loop\" namespace=moby pid=5830 runtime=io.containerd.runc.v2\ntime=\"2024-03-12T14:32:51+08:00\" level=warning msg=\"failed to read init pid file\" error=\"open /run/containerd/io.containerd.runtime.v2.task/moby/22fd6cedcf87252ef2585db60da56d1692f7aa64d4d3f7d77df51f1342931833/init.pid: no such file or directory\" runtime=io.containerd.runc.v2\n"
Mar 12 14:32:51 VM-0-4-centos containerd[1900]: time="2024-03-12T14:32:51.088573206+08:00" level=error msg="copy shim log" error="read /proc/self/fd/12: file already closed"

os log

tail -f /var/log/messages

Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered blocking state
Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered disabled state
Mar 12 14:34:39 VM-0-4-centos kernel: device vethd6bc28b entered promiscuous mode
Mar 12 14:34:39 VM-0-4-centos kernel: IPv6: ADDRCONF(NETDEV_UP): vethd6bc28b: link is not ready
Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered blocking state
Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered forwarding state
Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered disabled state
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.211561646+08:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.211711242+08:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.211733432+08:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.211880469+08:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.runtime.v2.task/moby/0acc1a759f03aeda353f36a467136c0fb0a0c660655ebca9bc7f8b2969b19e09 pid=5864 runtime=io.containerd.runc.v2
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.232562374+08:00" level=info msg="shim disconnected" id=0acc1a759f03aeda353f36a467136c0fb0a0c660655ebca9bc7f8b2969b19e09
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.232627552+08:00" level=warning msg="cleaning up after shim disconnected" id=0acc1a759f03aeda353f36a467136c0fb0a0c660655ebca9bc7f8b2969b19e09 namespace=moby
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.232638892+08:00" level=info msg="cleaning up dead shim"
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.238766721+08:00" level=warning msg="cleanup warnings time=\"2024-03-12T14:34:39+08:00\" level=info msg=\"starting signal loop\" namespace=moby pid=5886 runtime=io.containerd.runc.v2\ntime=\"2024-03-12T14:34:39+08:00\" level=warning msg=\"failed to read init pid file\" error=\"open /run/containerd/io.containerd.runtime.v2.task/moby/0acc1a759f03aeda353f36a467136c0fb0a0c660655ebca9bc7f8b2969b19e09/init.pid: no such file or directory\" runtime=io.containerd.runc.v2\n"
Mar 12 14:34:39 VM-0-4-centos containerd: time="2024-03-12T14:34:39.238954256+08:00" level=error msg="copy shim log" error="read /proc/self/fd/12: file already closed"
Mar 12 14:34:39 VM-0-4-centos dockerd: time="2024-03-12T14:34:39.239456785+08:00" level=error msg="stream copy error: reading from a closed fifo"
Mar 12 14:34:39 VM-0-4-centos dockerd: time="2024-03-12T14:34:39.239533924+08:00" level=error msg="stream copy error: reading from a closed fifo"
Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered disabled state
Mar 12 14:34:39 VM-0-4-centos kernel: device vethd6bc28b left promiscuous mode
Mar 12 14:34:39 VM-0-4-centos kernel: docker0: port 1(vethd6bc28b) entered disabled state
Mar 12 14:34:39 VM-0-4-centos dockerd: time="2024-03-12T14:34:39.278416306+08:00" level=error msg="Handler for POST /v1.44/containers/0acc1a759f03aeda353f36a467136c0fb0a0c660655ebca9bc7f8b2969b19e09/start returned error: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: can't get final child's PID from pipe: EOF: unknown"