Network creation fails with Error: 100: Key not found (/contiv/oper)

[mapuri]

This issue tracks an error when trying to create a network the network creation fails in the backend (with error logs pasted below). I have a workspace updated as of Feb 3, 2015 (commit: 005697b)

+++++ netdcli ++++
root@netplugin-node1:/opt/golang/src/github.com/contiv/netplugin# netdcli -oper create -construct network -tag 0 -subnet 1.1.1.0/24 infra
root@netplugin-node1:/opt/golang/src/github.com/contiv/netplugin#

++++ netplugin logs +++++
root@netplugin-node1:/opt/golang/src/github.com/contiv/netplugin# netplugin --host-label="host1"
2015/02/03 17:44:16 Received event for key: /contiv/config/nets/infra
2015/02/03 17:44:16 Network operation create failed. Error: 100: Key not found (/contiv/oper) [7]

[mapuri]

The error seems to prop from attempt to read operational global state (gstate.Oper) in CreateNetwork(), so it assumes that config global state (gstate.Config) has already been created before which was missing in my case and hence the error.

@jainvipin, just wanted to confirm if this is intended. In other words do you assume that global state will be be created before the networks and endpoints can be created? This can happen during netplugin init etc.

Also while working with multi-host stuff I saw that all netplugins try to act on global state and create the operational state, I think we might need a notion of per-host global state?

[jainvipin]

@mapuri - yes reading gstate during network creation is intended to facilitate auto allocation of subnet. And because subnet pool is shared across multiple hosts, the state is intended to be global across the cluster, not per host.
Having said that, I do see a requirement to have host level operational state rather than truly global state, like FreeLocalVlans that is put in gstate for now, but really global to the host, not cluster.

[mapuri]

@jainvipin

yes reading gstate during network creation is intended to facilitate auto allocation of subnet.

ok, I was confirming that since this ties the order of global-state creation and network creation if it was intended. I guess it should be fine to assume that global state be published before any networks or endpoints are published.

I do see a requirement to have host level operational state rather than truly global state, like FreeLocalVlans that is put in gstate for now

One issue with just global operational state per host about resource consumption (and global config state for cluster about the pools etc) is that it requires every host to read every other host's operation state in their resource allocation logic.

One solution might be to have, the resource allocation happen at a central entity like a resource scheduler. Or alternatively, have the central entity create per host global config state. The latter approach however requires resource allocation at two levels and makes things unnecessarily complex.

I think we might need to hash this out. To not digress from the main thread I will close the main issue and open new one for global state management.