Dockerhost not working

[jsoyee]

It's weird that after start netplugin-node1 successfully, the br-em1 got deleted.

[root@netplugin-02 netplugin]# /bin/systemctl restart docker.service
[root@netplugin-02 netplugin]# brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.56847afe9799 no
[root@netplugin-02 netplugin]# make start-dockerdemo
scripts/dockerhost/start-dockerhosts
scripts/dockerhost
Num nodes = 2
Trying to pull repository docker.io/ubuntu ...
07f8e8c5e660: Download complete
e9e06b06e14c: Download complete
a82efea989f9: Download complete
37bea4ee0c81: Download complete
Status: Image is up to date for docker.io/ubuntu:latest
mkdir: cannot create directory ‘/tmp/ubuntu_image’: File exists
Starting netplugin-node1
6d604a6a34f3b66c4f998c5f73ccc7443c08be3048bedbe0955c197d5661cb91
172.17.0.1 netplugin-node1
IP address = 192.168.2.10/32
mount: cgroup already mounted or cpu busy
mount: according to mtab, cgroup is mounted on /sys/fs/cgroup
mount: cgroup already mounted or cpuacct busy
mount: according to mtab, cgroup is mounted on /sys/fs/cgroup
Starting Docker: docker
...done.
Starting netplugin-node2
03f5a45b84cf66fd3a2adc297f4d820b6ff8d74706f789a0f5321b8bc684d9e9
FATA[0001] Error response from daemon: Cannot start container 03f5a45b84cf66fd3a2adc297f4d820b6ff8d74706f789a0f5321b8bc684d9e9: no such network interface
FATA[0000] Error response from daemon: Container netplugin-node2 is not running
FATA[0000] Error response from daemon: Container netplugin-node2 is not running
bridge br-em1 does not exist!
RTNETLINK answers: No such process
nsenter: neither filename nor target pid supplied for ns/net
nsenter: neither filename nor target pid supplied for ns/net
IP address = 192.168.2.11/32
nsenter: neither filename nor target pid supplied for ns/net
nsenter: neither filename nor target pid supplied for ns/net
FATA[0000] Error response from daemon: Container netplugin-node2 is not running
FATA[0000] Error response from daemon: Container netplugin-node2 is not running
Starting etcd on netplugin-node1
Starting etcd on netplugin-node2
FATA[0000] Error response from daemon: Container netplugin-node2 is not running
[root@netplugin-02 netplugin]# brctl show
bridge name bridge id STP enabled interfaces
[root@netplugin-02 netplugin]#

[mapuri]

I think this issue sounds similar to what I debugged with recent docker-in-docker demo environment changes for netplugin.

The dockerhost demo environment requires ovs to be installed on the host (i.e. the baremetal or the vm in which the demo is being setup) other wise it seems to start fiddling with the linux bridge on the host/vm.

Can you confirm if following steps help you recover?

• install ovs
• restart docker
• make start-dockerdemo

[jsoyee]

Not lucky, still same after i have OVS installed on my VM. ;-(

On Sat, May 9, 2015 at 1:00 AM, Madhav Puri [email protected]
wrote:

I think this issue sounds similar to what I debugged with recent
docker-in-docker demo environment changes for netplugin.

The dockerhost demo environment requires ovs to be installed on the host
(i.e. the baremetal or the vm in which the demo is being setup) other wise
it seems to start fiddling with the linux bridge on the host/vm.

Can you confirm if following steps help you recover?

• install ovs
• restart docker
• make start-dockerdemo

—
Reply to this email directly or view it on GitHub
#62 (comment).

[mapuri]

ok, let me check the scripts.

Can you tell me know your vm environment (linux flavor etc) as well, I can try reproduce it.

We do run docker-in-docker tests on a Ubuntu 14.10 based machine, which seem to be passing.

ping @sachja in case he has seen anything similar during his tests.

[jsoyee]

Thanks, here is my docker info
[root@netplugin-02 ~]# docker info
Containers: 2
Images: 43
Storage Driver: devicemapper
Pool Name: docker-253:1-117470914-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: xfs
Data file:
Metadata file:
Data Space Used: 1.717 GB
Data Space Total: 107.4 GB
Data Space Available: 105.7 GB
Metadata Space Used: 2.916 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.145 GB
Udev Sync Supported: true
Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Kernel Version: 3.10.0-229.1.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
CPUs: 2
Total Memory: 3.703 GiB
Name: netplugin-02.novalocal
ID: LVXQ:7Q2B:6Y2O:M26V:S4YS:DGX2:WRXC:XCMP:SRXE:HBGI:O3TP:MDUY

I will try ubuntu as well. will let you know the result

On Sat, May 9, 2015 at 9:25 AM, Madhav Puri [email protected]
wrote:

ok, let me check the scripts.

Can you tell me know your vm environment (linux flavor etc) as well, I can
try reproduce it.

We do run docker-in-docker tests on a Ubuntu 14.10 based machine, which
seem to be passing.

ping @sachja https://github.com/sachja in case he has seen anything
similar during his tests.

—
Reply to this email directly or view it on GitHub
#62 (comment).

[sachja]

As Madhav said we had hit the br-em1 delete issue due to ovs not installed.
Did you restart docker before trying this again?

sudo service docker restart

I have tried this on centos baremetal, ubuntu baremetal and ubuntu VM on a
windows laptop.

sachin

On Fri, May 8, 2015 at 6:25 PM, Madhav Puri [email protected]
wrote:

ok, let me check the scripts.

Can you tell me know your vm environment (linux flavor etc) as well, I can
try reproduce it.

We do run docker-in-docker tests on a Ubuntu 14.10 based machine, which
seem to be passing.

ping @sachja https://github.com/sachja in case he has seen anything
similar during his tests.

—
Reply to this email directly or view it on GitHub
#62 (comment).

[jsoyee]

Yes, i tried what Madhav mentioned, I installed OVS(version 2.3.1) and then
restart docker services , it still not working, and then i reinstalled
docker , still not lucky .

On Sat, May 9, 2015 at 9:46 AM, sachja [email protected] wrote:

As Madhav said we had hit the br-em1 delete issue due to ovs not installed.
Did you restart docker before trying this again?

sudo service docker restart

I have tried this on centos baremetal, ubuntu baremetal and ubuntu VM on a
windows laptop.

sachin

On Fri, May 8, 2015 at 6:25 PM, Madhav Puri [email protected]
wrote:

ok, let me check the scripts.

Can you tell me know your vm environment (linux flavor etc) as well, I
can
try reproduce it.

We do run docker-in-docker tests on a Ubuntu 14.10 based machine, which
seem to be passing.

ping @sachja https://github.com/sachja in case he has seen anything
similar during his tests.

—
Reply to this email directly or view it on GitHub
#62 (comment).

—
Reply to this email directly or view it on GitHub
#62 (comment).

[mapuri]

@jsoyee

I tried make start-dockerdemo on a Ubuntu vm and it went well. Somehow centos based vm didn't work that well for me as I hit some other issues with my Vagrant.

I will close this issue as the script seems to work for Ubuntu. Let me know in case you hit any other issues after moving to Ubuntu.

I have also added a few checks (as part of #65) in the script for docker based setups to error out gracefully in case some prerequisites are not met or it is run in an untested Linux. We have mostly tried this on Ubuntu till now but it would be good to have it work on other *nix flavors as well. Happy to take pull requests!