kvm interface, auto scheduler

Author: lowang-bh

README.md

@@ -36,14 +36,35 @@
 - VIF: virtual  interface, the eth on VM
 - SR:  storage repository, the storage pool on server
 
-# Create KVM VMS
-  1. Auto schedual the target vm to a available server, this is only available on dev env
+# Create KVM VMs Auto Schedule
 
-  > lain3_node.py --role=rolename --cluster=<test|xyz|kvm>
+  Note: `schedule_node.py` can auto schedule the wanted vm to a available server  
+  you can list the supported role with `--list-roles`, if there is no role you want, you can define a new one  
+  in `etc/Nodeconfig.json`, each role is a dict defined the cpu, memory, etc. You can define `k8s-master`,  
+  `k8s-node`, `etcd` template as you want.  
+  
+  for example:
+  
+  ```
+      "jenkins": {
+        "cpu": 8,
+        "memory": 32,
+        "template": "template",
+        "disk_size": 100,
+        "add_disk_num": 2
+    }
+  ```
+
+##  1. Auto schedual the target vm to a available server
+
+  > schedule_node.py --role=rolename --cluster=<dev|test|prod>
+  
+  > Note: user can define the vm-name rule at function `generate_vmname_key(role, cluster)` in file `lib/Utils/schedule.py`
 
-  2. manually given the target vm-name, host and vm-ip. Product env should use this command
+##  2. manually given the target vm-name, host and vm-ip.
 
-  > lain3_node.py --role=rolename --name=new_vm_name --host=hostip --ip=vm_ip
+  > schedule_node.py --role=rolename --name=new_vm_name --host=hostip --ip=vm_ip
+  
 # Create multiple VMs from xml config
 - `setup_vms.py --validate xmlFile`     Validate the given xml file
 - `setup_vms.py --create   xmlFile`     Do the create up according to the xml file
@@ -102,13 +123,13 @@ Create VMs with xml just need to write a xml config following the xml-example. B
   - `--list-network`        List the bridge/switch network in the host
   - `--list-SR`             List the storage repository infor in the host
 
-#####  1). <b>**Create a new VM with a template:**<b>
+#####  2.1). <b>**Create a new VM with a template:**<b>
   - `-c VM_NAME, --create=VM_NAME`                        Create a new VM with a template.
   - `-t TEMPLATE, --templ=TEMPLATE`                       Template used to create a new VM.
 
-    > create_vm.py -c "test_vm" -t "CentOS 7.2 for Lain"
+    > create_vm.py -c "test_vm" -t "CentOS 7.2 Template"
 
-#####  2). <b>**Create a new VM with a given IP: if a IP specified**<b>
+#####  2.2). <b>**Create a new VM with a given IP: if a IP specified**<b>
   - `--vif=VIF_INDEX`      The index of virtual interface on which configure will be
   - `--device=DEVICE`      The target physic NIC name with an associated network VIF attach(ed) to
   - `--network=NETWORK`    The target bridge/switch network which VIF connect(ed) to
@@ -122,15 +143,15 @@ Create VMs with xml just need to write a xml config following the xml-example. B
     **neither *--device* nor *--network*, the default manage network will be used**
     > create_vm.py -c "test2" -t "CentOS 7.2 template" --ip=192.168.1.100 --vif=0
 
-#####  3). <b>**Create a new VM with given max cpu cores and current cpu cores:**<b>
+#####  2.3). <b>**Create a new VM with given max cpu cores and current cpu cores:**<b>
   - `--cpu-max=MAX_CORES`   Config the max VCPU cores.
   - `--cpu-cores=CPU_CORES` Config the number of startup VCPUs for the new created VM
 
     > create_vm.py -c "test2" -t "CentOS 7.2 template" --cpu-core=2 --cpu-max=4
 
     The max cpu cores can be configured when VM is power off only, and it affect the upper limit when set the cpu cores lively
 
-#####  4). <b>**Create a new VM with memory size:**<b>
+#####  2.4). <b>**Create a new VM with memory size:**<b>
   - `--memory=MEMORY_SIZE`  Config the target memory size in GB.
   - `--min-mem=MIN_MEMORY`  Config the min static memory size in GB.
   - `--max-mem=MAX_MEMORY`  Config the max static memory size in GB.
@@ -144,7 +165,7 @@ Create VMs with xml just need to write a xml config following the xml-example. B
   - `--max-mem will set the max static memory `
     > create_vm.py -c "test2" -t "CentOS 7.2 template" --memory=2 --max-mem=4
 
-##### 5).Add new disk to VM, the storage_name is choosed from *--list-SR*
+##### 2.5).Add new disk to VM, the storage_name is choosed from *--list-SR*
   - `--add-disk=DISK_SIZE`  The disk size(GB) add to the VM
   - `--storage=STORAGE_NAME` The storage location where the virtual disk put
     > create_vm.py "test1"--add-disk=2 --storage=data2
@@ -165,7 +186,7 @@ Create VMs with xml just need to write a xml config following the xml-example. B
   - `--list-network`        List the bridge/switch network in the host
   - `--list-SR`             List the storage repository information in the host
 
-##### 1). <b>**Config a VM's interface, add a VIF, delete a VIF, config a VIF(will delete old one if exist, otherwise create it newly), and the *--ip*, *--device*, *--network* is same as that when create vm**<b>
+##### 4.1). <b>**Config a VM's interface, add a VIF, delete a VIF, config a VIF(will delete old one if exist, otherwise create it newly), and the *--ip*, *--device*, *--network* is same as that when create vm**<b>
   - `--add-vif=ADD_INDEX`   Add a virtual interface device to guest VM
   - `--del-vif=DEL_INDEX`   Delete a virtual interface device from guest VM
   - `--vif=VIF_INDEX`      Configure on a virtual interface device
@@ -174,17 +195,17 @@ Create VMs with xml just need to write a xml config following the xml-example. B
 
     > config_vm.py "test1"  --add-vif=1 --device=eth1 --ip=192.168.1.200 --netmask=255.255.255.0
 
-##### 2). <b>**config a VM' cpu when it is running**<b>
+##### 4.2). <b>**config a VM' cpu when it is running**<b>
   - `--cpu-cores=CPU_CORES` Config the VCPU cores lively for a running VM or the number of startup VCPUs for a halted VM
 
       > config_vm.py "test1" --cpu-core=4
 
-##### 3). <b>**config a VM' memory when it is running**<b>
+##### 4.3). <b>**config a VM' memory when it is running**<b>
   - `--memory=MEMORY_SIZE`  Config the target memory size in GB.
 
     > config_vm.py "test1" --memory=1
 
-##### 4).Add new disk to VM, the storage_name is choosed from *--list-SR*
+##### 4.4).Add new disk to VM, the storage_name is choosed from *--list-SR*
   - `--add-disk=DISK_SIZE`  The disk size(GB) add to the VM
   - `--storage=STORAGE_NAME` The storage location where the virtual disk put
     > config_vm.py "test1"--add-disk=2 --storage=data2

etc/lain3Nodeconfig.json etc/Nodeconfig.json

@@ -1,15 +1,15 @@
 {
-    "master": {
+    "k8s-master": {
         "cpu": 8,
         "memory": 16,
         "template": "k8s-template"
     },
-    "ingress": {
+    "k8s-ingress": {
         "cpu": 8,
         "memory": 16,
         "template": "k8s-template"
     },
-    "node": {
+    "k8s-node": {
         "cpu": 16,
         "memory": 48,
         "template": "k8s-template"
@@ -30,5 +30,12 @@
        "cpu": 4,
        "memory": 8,
        "template": "nfs-template"
+    },
+    "gluster": {
+        "cpu": 4,
+        "memory": 6,
+        "template": "nfs-template",
+        "disk_size": 50,
+        "add_disk_num": 2
     }
 }

lib/Log/logging_server.py

@@ -4,13 +4,13 @@
  File Name: constans.py
  Author: longhui
  Created Time: 2019-04-24 11:07:34
- Description: lain3 node for k8s cluster default config
+ Description: different role of node with default config
 """
 template_dict = {
 
-    "master": {"cpu": 8, "memory": 16, "template": "k8s-template"},
-    "ingress": {"cpu": 8, "memory": 16, "template": "k8s-template"},
-    "node": {"cpu": 16, "memory": 32, "template": "k8s-template", "disk_size": 200, "add_disk_num": 1},
+    "k8s-master": {"cpu": 8, "memory": 16, "template": "k8s-template"},
+    "k8s-ingress": {"cpu": 8, "memory": 16, "template": "k8s-template"},
+    "k8s-node": {"cpu": 16, "memory": 32, "template": "k8s-template", "disk_size": 200, "add_disk_num": 1},
     "etcd": {"cpu": 4, "memory": 8, "template": "k8s-template", "disk_size": 100, "add_disk_num": 1},
     "jenkins": {"cpu": 8, "memory": 32, "template": "template", "disk_size": 100, "add_disk_num": 2},
     }

lib/Utils/constans.py

@@ -14,7 +14,7 @@ class XenVirtDriverTestCase(unittest.TestCase):
 
     def setUp(self):
         self.virt_driver = XenVirtDriver(hostname="192.168.1.2", user="root", passwd="123456")
-        self.vms = ['t2-dev16', 't2-jenkins-lain2', 't2-dev24', 't2-dev23', 't2-dev25', 't2-dev26', 't2-dev22']
+        self.vms = ['t2-dev16', 't2-jenkins', 't2-dev24', 't2-dev23', 't2-dev25', 't2-dev26', 't2-dev22']
 
     def tearDown(self):
         self.virt_driver.delete_handler()
@@ -29,7 +29,7 @@ def test_get_vm_list(self):
             self.assertEqual(vm in dst_vms, True, "%s not there" % vm)
 
     def test_get_template_list(self):
-        templates = ["CentOS 7.2 for Lain"]
+        templates = ["CentOS 7.2 Template"]
         dst_templates = self.virt_driver.get_templates_list()
         self.assertNotEqual(dst_templates, [], "get_templates_list() return []")
         for templ in templates:

lib/Val/Xen/virt_driver_xen_test.py

@@ -1,7 +1,7 @@
 six
 ipaddress
 requests
-urllib3==1.23
+urllib3==1.24.2
 chardet
 certifi
 idna

requirements.txt

@@ -1,10 +1,10 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 """
- File Name: lain3_node.py
+ File Name: schedule_node.py
  Author: longhui
  Created Time: 2019-04-24 10:44:45
- Description: Add  lain3 node for k8s cluster 
+ Description: schedule different type of node to server, for k8s, jenkins, etcd cluster and so on 
 """
 import json
 import os
@@ -19,9 +19,9 @@
 
 if __name__ == "__main__":
     usage = """usage: %prog [options] arg1 arg2\n
-        lain3_node.py --role=rolename --cluster=[test|xyz|kvm]
-        lain3_node.py --role=rolename --name=new_vm_name --host=hostip --ip=vm_ip
-        lain3_node.py --list-roles
+        schedule_node.py --role=rolename --cluster=[test|xyz|kvm]
+        schedule_node.py --role=rolename --name=new_vm_name --host=hostip --ip=vm_ip
+        schedule_node.py --list-roles
         """
     parser = OptionParser(usage=usage)
     parser.add_option("--role", dest="rolename", help="Config the target role of vm to create.")
@@ -40,7 +40,7 @@
 
     config_dict = template_dict
     try:
-        config_file = os.path.join(os.path.dirname(__file__), "../etc/lain3Nodeconfig.json")
+        config_file = os.path.join(os.path.dirname(__file__), "../etc/Nodeconfig.json")
         with open(config_file, "r") as f:
             try:
                 config = json.load(f)
@@ -69,7 +69,7 @@
         log.fail("Does not support role: %s", options.rolename)
         exit(1)
 
-    # if "--cluster" is given, use  command "lain3_node.py --role=rolename --cluster=cluster-name" 
+    # if "--cluster" is given, use  command "schedule_node.py --role=rolename --cluster=cluster-name" 
     if options.cluster is not None:
         cluster_name = options.cluster
         #  when no input, find default server and default vmname and IP
@@ -90,7 +90,7 @@
             exit(1)
 
     else:
-        # use command "lain3_node.py --role=rolename  --name=new_vm_name --host=hostip --ip=vm_ip"
+        # use command "schedule_node.py --role=rolename  --name=new_vm_name --host=hostip --ip=vm_ip"
         if options.hostip is None or options.new_vm_name is None or options.vm_ip is None:
             log.fail("'--name=new_vm_name --host=hostip --ip=vm_ip' is required when with no '--cluster'.")
             exit(1)