Docker exits in host network mode #56

lilveg · 2020-05-27T10:34:58Z

Hello!

I want to use OpenWRT on an embedded device running docker. I have observed the following behaviour both on my development laptop and on the target device:

$ time docker run --network host openwrtorg/rootfs:x86-64-19.07.2
/etc/preinit: line 6: can't create /sys/devices/system/cpu/microcode/reload: Read-only file system
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level

real	0m3,011s
user	0m0,045s
sys	0m0,028s

I am using time to give additional data, the same thing happens if I run without it. Also, I believe I need to use the --cap-add NET_ADMIN parameter to allow OpenWRT to make changes, but it does not seem to make a difference for now.

If I start the container using docker run --network host -it openwrtorg/rootfs:x86-64-19.07.2 ash I get a shell and I'm able to do some things, but important services like ubus are not running.

Is this a problem with init? /etc/init.d/rcS doesn't exist even though it's referenced by /etc/inittab. Any help locating the source of the problem would be appreciated!

The text was updated successfully, but these errors were encountered:

shivarammysore · 2020-07-16T23:54:47Z

Any guidance on what is wrong?

$ docker run --privileged --network="host" -it openwrtorg/rootfs:x86-64-19.07.3 ash
BusyBox v1.30.1 () built-in shell (ash)
/ # /sbin/init
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
1
/ # /sbin/procd
^C/ # exit

aparcar · 2020-07-17T04:23:01Z

@shivarammysore I don't really understand your error message.

shivarammysore · 2020-07-17T04:43:47Z

@aparcar I am trying to run OpenWRT on a docker container. When I run it as:

$ sudo docker run -d -t --name openwrt  --privileged  --network="host" openwrtorg/rootfs:x86-64-19.07.3
$ docker ps -a
CONTAINER ID        IMAGE                              COMMAND             CREATED             STATUS                      PORTS               NAMES
6bf827da90ca        openwrtorg/rootfs:x86-64-19.07.3   "/sbin/init"        20 seconds ago      Exited (1) 16 seconds ago                       openwrt

$ docker logs openwrt
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level

Question is if the above is not the right way to run OpenWRT in a container, then what is the correct procedure?

Thanks

aparcar · 2020-07-17T04:48:03Z

Why would you want to run it as privileged? OpenWrt does some interface resets on init, so it likely hangs somewhere there

shivarammysore · 2020-07-17T04:56:24Z

@aparcar
If you don't use privileged option, you will get:

Failed to resize receive buffer: Operation not permitted
/etc/preinit: line 6: can't create /sys/devices/system/cpu/microcode/reload: Read-only file system
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level

aparcar · 2020-07-17T05:40:22Z

Maybe you're missing -i? Or did you try to login via SSH?

ser@dawn:~$ sudo docker run -it --rm openwrtorg/rootfs:x86-64-19.07.3
Unable to find image 'openwrtorg/rootfs:x86-64-19.07.3' locally
x86-64-19.07.3: Pulling from openwrtorg/rootfs
a5a5938fa80f: Pull complete 
Digest: sha256:2a7eea411a12c882705bc8fecb2810cd7c0e795e3eca30a3e546dc2bebda547d
Status: Downloaded newer image for openwrtorg/rootfs:x86-64-19.07.3
Failed to resize receive buffer: Operation not permitted
/etc/preinit: line 6: can't create /sys/devices/system/cpu/microcode/reload: Read-only file system
ip: RTNETLINK answers: Operation not permitted
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
ip: can't send flush request: Operation not permitted
ip: SIOCSIFFLAGS: Operation not permitted
Please press Enter to activate this console.



BusyBox v1.30.1 () built-in shell (ash)

/bin/ash: can't access tty; job control turned off
  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 19.07.3, r11063-85e04e9f46
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@f92852f80cae:/# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=59 time=5.135 ms
64 bytes from 1.1.1.1: seq=1 ttl=59 time=5.843 ms
64 bytes from 1.1.1.1: seq=2 ttl=59 time=4.069 ms
^C64 bytes from 1.1.1.1: seq=3 ttl=59 time=5.371 ms

shivarammysore · 2020-07-17T06:00:04Z

@aparcar thanks for the hint. -i is the interactive shell mode. Yes it does work:

sudo docker run --privileged -p 80:80 -p 443:443 -it --rm openwrtorg/rootfs:x86-64-19.07.3
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level

Please press Enter to activate this console.



BusyBox v1.30.1 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 19.07.3, r11063-85e04e9f46
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# netstat -lpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      559/uhttpd
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      347/dnsmasq
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      384/dropbear
tcp        0      0 :::80                   :::*                    LISTEN      559/uhttpd
tcp        0      0 fe80::42:acff:fe11:2:53 :::*                    LISTEN      347/dnsmasq
tcp        0      0 ::1:53                  :::*                    LISTEN      347/dnsmasq
tcp        0      0 :::22                   :::*                    LISTEN      384/dropbear
udp        0      0 127.0.0.1:53            0.0.0.0:*                           347/dnsmasq
udp        0      0 fe80::42:acff:fe11:2:53 :::*                                347/dnsmasq
udp        0      0 ::1:53                  :::*                                347/dnsmasq
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node PID/Program name    Path
unix  2      [ ACC ]     STREAM     LISTENING     26470130 75/ubusd            /var/run/ubus.sock
root@OpenWrt:/# halt

Now, If I run

$ curl http://localhost 
curl: (56) Recv failure: Connection reset by peer

I also tried to reach the OpenWRT console via browser and it does not work.
What I also confirmed is that --network=host is also not working. The instance just exists.

shivarammysore · 2020-07-17T06:11:14Z

I see that

root@OpenWrt:/# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
ping: sendto: Network unreachable
root@OpenWrt:/# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
root@OpenWrt:/#

did you change any network settings when you configured the docker instance?

Thanks

aparcar · 2020-07-17T06:41:16Z

No. But I don't let it use the host network. What is you use case? Running docker privileged and using host networks sound you should install OpenWrt in the first place?

shivarammysore · 2020-07-17T06:57:08Z

My use case is to run OpenWRT on Fedora CoreOS. /

…

On Jul 16, 2020, at 11:41 PM, Paul Spooren ***@***.***> wrote: No. But I don't let it use the host network. What is you use case? Running docker privileged and using host networks sound you should install OpenWrt in the first place? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

shivarammysore · 2020-07-17T16:16:48Z

@aparcar When we run OpenWRT as a container, without the --network=host option, we will not have access to the physical interfaces on the box. In a standard mode without host networking, we will have just the eth0 interface on the container and hence, services cannot be used or accessed. I have more detailed documentation on how host networking works @ https://github.com/shivarammysore/ovs/blob/master/docs/docker_networking.rst

root@OpenWrt:/etc/config# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether e6:ee:52:08:bf:15 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e4ee:52ff:fe08:bf15/64 scope link 
       valid_lft forever preferred_lft forever

I have on my host system a host of physical network interfaces.

$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0d:48:59:00:c0 brd ff:ff:ff:ff:ff:ff
    altname enp2s0
    inet 10.20.30.208/16 brd 10.20.255.255 scope global dynamic noprefixroute eth0
       valid_lft 4999sec preferred_lft 4999sec
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c1 brd ff:ff:ff:ff:ff:ff
    altname enp3s0
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c2 brd ff:ff:ff:ff:ff:ff
    altname enp4s0
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c3 brd ff:ff:ff:ff:ff:ff
    altname enp5s0
6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c4 brd ff:ff:ff:ff:ff:ff
    altname enp10s0f0
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c5 brd ff:ff:ff:ff:ff:ff
    altname enp10s0f1
8: eth6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c6 brd ff:ff:ff:ff:ff:ff
    altname enp11s0f0
9: eth7: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0d:48:59:00:c7 brd ff:ff:ff:ff:ff:ff
    altname enp11s0f1
10: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b4:82:c5:58:0b:70 brd ff:ff:ff:ff:ff:ff
    altname wlp9s0

I want to use these network interfaces with OpenWRT. This will be similar to the work I have done for Open vSwitch - https://github.com/shivarammysore/ovs

shivarammysore mentioned this issue Jul 18, 2020

Simpler version without interfaces oofnikj/docker-openwrt#6

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker exits in host network mode #56

Docker exits in host network mode #56

lilveg commented May 27, 2020

shivarammysore commented Jul 16, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020 via email

shivarammysore commented Jul 17, 2020

Docker exits in host network mode #56

Docker exits in host network mode #56

Comments

lilveg commented May 27, 2020

shivarammysore commented Jul 16, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

shivarammysore commented Jul 17, 2020

aparcar commented Jul 17, 2020

shivarammysore commented Jul 17, 2020 via email

shivarammysore commented Jul 17, 2020