Makefile, Vagrantfile: refactor of build system

[erikh]

This implements several features for the build system:

• Builds happen on the VM in a consistent way, so they can happen on non linux
  platforms.
• New builder images include:
  • First class support of 4.0 (Ubuntu) and 4.1 (CentOS) mainline kernels
  • Docker 1.8.1
  • systemd (with proxy support)

Signed-off-by: Erik Hollensbe [email protected]

[erikh]

is CI down?

[contiv]

build status: https://25b17de9.ngrok.com/job/Pull-Request%20Build%20Master/43/console

[mapuri]

I think we should stop the vm once build is done, otherwise it will stay running.

[erikh]

the problem with that is that each build will trigger a vm restart -- this is very slow for a build/test cycle.

[mapuri]

ok, but if user were to just do make build and stop. They won't know that we left a vm running behind. And If they decide to yank out their workspace and pull a new one, this vm will stay there consuming resources.

I might be coming out a bit paranoid but cleanup somewhat seems a lot safer at expense of spending some time :)

[erikh]

I don't see how this is any different from typical vagrant usage -- if someone does that without stopping the VM, they will be stuck with the same solution, pkill VBoxManage roughly.

I understand your desire but we have to figure out whether keeping the build time down or protecting the user is more important here. I don't know if there's a way to get the best of both worlds.

[mapuri]

I don't see how this is any different from typical vagrant usage

Yes, this is not different in behavior. The only difference is wrt user's experience. In case of doing vagrant up and then deleting the workspace we can consider it to be error on user's part if it leaves behind a vm. But if a user does make build and then delete their workspace which leaves behind a vm, it is not an error on their part.

You are right there is no easy way to achieve both in this case i.e. protect the user or improved build time. We can may be print a informative message about the running vm after build to alleviate this a bit. For now, I am ok starting with what we have here as it does solves the builds on mac.

ping @jainvipin @shaleman as well to get their thoughts.

[mapuri]

LGTM except for the closure on make build behavior, it would be good to get an ok from others as well.

Otherwise good to merge after sanity passes.

[erikh]

decided:

make demo will do an implicit build.

[jainvipin]

@erikh +1
Overall a great step for making it easy for anyone to try this out. Once checked in, I want to be the first user :-)

• I saw you and @mapuri talking about cleanup increasing the time. Do we know how much is the increase per test run?
• How does it affect developer's build experience? Does he use the same VM based build? If so, how long does it take, it was taking few seconds earlier.
• I also feel that we need to update docs to suggest new ways of using netplugin: 4 step process: 1) git clone; 2) make demo (with netmaster/netplugin included); 3) Run netdcli with an intent; 4) launch containers, and see the connectivity.

[erikh]

1. The cleanup itself is cheap; it takes 20-30 seconds to build the whole netplugin suite. The downside to this newer approach is that to avoid object format conflicts we're going to need to bring up and down a lot more VMs. Ultimately this is the majority of the build time, adn it's not too unpleasant on modern systems.
2. It makes the entire build more consistent at the expense of speed in testing. Yes, the build always comes from the VM now. It can take up to a minute or two to bring the VM up, but we are working on different approaches to making building easier on the guest VMs themselves.
3. I have made a task for this and will do so later today.

[erikh]

@mapuri this is completing on everything but DIND, and we recently discussed disabling those tests.

Perhaps now is the time? :)

[mapuri]

@erikh, for this particular PR if it's just the known ovs/timing failure I think we should let his one go in as it's a known failure.

For disabling the DIND I am fine turning them off. I am just wondering, if we should watch for a little bit more into recent and future sanity failures that relate to the DIND environment. Somehow I have a feeling that this failure is one off and can be hit in any environment. WDYT?

also ping @jainvipin, to conveniently punt the decision to him :)

[erikh]

The test failures are DIND-only now. The heisenbug in the ovs code can be handled out-of-band.

Happy to punt to @jainvipin too :)

[jainvipin]

@mapuri, @erikh: let's go without dind, which is not a production scenario we'd like to support anyways.

[erikh]

Ok, then I believe this can be merged. If nobody objects I will tomorrow.

[mapuri]

sure, SGTM. I have disabled DIND sanity in our jenkins CI.

We can eventually cleanup the DIND related code as a separate PR. I will file an issue for the same.