Update CLI instructions

Author: niloysh

README.md

@@ -165,7 +165,14 @@ In this deployment, Open5GS is configured to support two network slices. Each sl
 ### Deploy Open5GS components
 To deploy Open5GS, apply the Kubernetes manifest files in the open5gs/ directory. These manifest files define each Open5GS NF as a separate pod, allowing the platform to operate in a distributed fashion.
 
-Select one of the following deployment options based on your monitoring needs:
+Select one of the following deployment options based on your monitoring needs - **Standard Deployment** or **Deployment with Monarch**.
+
+> [!WARNING]
+> Deploy only one option at a time.
+
+> [!NOTE]
+> To switch deployments (e.g., from Standard to Monarch), first delete the current deployment with
+> `kubectl delete -k open5gs -n open5gs`. Then proceed with the other deployment option.
 
 #### 1. Standard Deployment:
 Deploys Open5GS network functions as separate pods for a distributed setup.

add-subscribers-using-cli.md

@@ -1,26 +1,28 @@
 # Add subscribers using CLI
 
-Using the CLI to add subscribers and slices is particularly efficient when dealing with multiple slices (e.g., 10 or more). This method enables batch additions and can be adjusted via the config.yaml file in the data directory.
+Using the CLI to add subscribers and slices is particularly efficient when dealing with multiple slices (e.g., 10 or more). This method enables batch additions and can be adjusted via the configuration options described below.
 
-## Configuration File (config.yaml)
-The default configuration (data/config.yaml) is shown below:
+## Configuration File (mongo-tools/generate-data.py)
+The default configuration (mongo-tools/generate-data.py) is shown below:
 
-```yaml
-NUM_SLICES: 2
-NUM_SUBSCRIBERS: 0 # simulated subscribers, one per slice e.g., ue_101, ue_201 etc.
-COTS_UE: false # Test SIMs used in COTS UE (Pixel 7 Pro)
-SIM_UE: true # 3 simulated UEs (ue1, ue2, ue3) for testing with UERANSIM
+```python
+DEFAULT_CONFIG = {
+    "NUM_SLICES": 2,  # Number of network slices to create
+    "NUM_COTS_SUBSCRIBERS": 0,  # Number of COTS subscribers
+    "NUM_SAMPLE_SUBSCRIBERS": 2,  # Number of simulated subscribers
+    "NUM_AUTO_GENERATED_SUBSCRIBERS": 0,  # Number of auto-generated subscribers (for MSD deployment only)
+}
 ```
 
 ### Types of Subscribers
 
 Our setup supports three types of subscribers:
 
-- **COTS_UE**: Refers to Commercial Off-The-Shelf UEs configured with programmable SIM cards, used with USRP-based gNBs. Currently, these are configured for the UWaterloo lab.
-- **SIM_UE**: Denotes simulated UEs, such as subscriber_1, subscriber_2, and subscriber_3, for testing with UERANSIM. By default:
+- **NUM_COTS_SUBSCRIBERS**: Refers to Commercial Off-The-Shelf UEs configured with programmable SIM cards, used with USRP-based gNBs. Currently, these are configured for the UWaterloo lab.
+- **NUM_SAMPLE_SUBSCRIBERS**: Denotes simulated UEs, such as subscriber_1, subscriber_2, and subscriber_3, for testing with UERANSIM. By default:
   - subscriber_1 and subscriber_3 connect to slice 1.
   - subscriber_2 connects to slice 2.
-- **NUM_SUBSCRIBERS**: Specifies the number of programmatically added simulated UEs. Each UE is assigned to slices in a round-robin fashion: For instance, if NUM_SUBSCRIBERS is 3 and NUM_SLICES is 2, the subscribers will be added as follows:
+- **NUM_AUTO_GENERATED_SUBSCRIBERS**: Specifies the number of automatically generated simulated UEs. Each UE is assigned to slices in a round-robin fashion: For instance, if NUM_SUBSCRIBERS is 3 and NUM_SLICES is 2, the subscribers will be added as follows:
   - Slice 1: subscriber_101
   - Slice 2: subscriber_201
   - Slice 1: subscriber_102
@@ -33,16 +35,20 @@ Our setup supports three types of subscribers:
 ## Adding subscribers
 Ensure your virtual environment is active as described in as described in [Set up a virtual environment](README.md#1-set-up-a-virtual-environment).
 
-1. Run the generate-data.py script, which reads data/config.yaml and generates two files: data/slices.yaml and data/subscribers.yaml.
+1. Run the generate-data.py script, generates two files: `data/slices.yaml` and `data/subscribers.yaml`.
 
     ```bash
     (venv) dev@workshop-vm:~/open5gs-k8s$ python mongo-tools/generate-data.py 
     ```
     You should see output similar to the one below.
     ```bash
-    2024-10-31 11:38:29 |     INFO | Creating 2 slices and 0 subscribers
-    2024-10-31 11:38:29 |     INFO | Creating slices and saving to data/slices.yaml
-    2024-10-31 11:38:29 |     INFO | Creating subscribers and saving to data/subscribers.yaml
+    2024-11-06 16:36:02 |     INFO | Loading existing data...
+    2024-11-06 16:36:02 |     INFO | Creating 2 slices ...
+    2024-11-06 16:36:02 |     INFO | Generating 0 new slices...
+    2024-11-06 16:36:02 |     INFO | Saving slices to data/slices.yaml
+    2024-11-06 16:36:02 |     INFO | Adding 2 sample subscribers ...
+    2024-11-06 16:36:02 |     INFO | Saving subscribers to data/subscribers.yaml
+    2024-11-06 16:36:02 |     INFO | Slice and subscriber creation complete.
     ```
 2. Run add-subscribers.py to add the generated subscribers.
 
@@ -54,9 +60,10 @@ Ensure your virtual environment is active as described in as described in [Set u
     ```bash
     2024-10-31 11:54:22 |     INFO | Added subscriber_1
     2024-10-31 11:54:22 |     INFO | Added subscriber_2
-    2024-10-31 11:54:22 |     INFO | Added subscriber_3
     ```
 
+> [!NOTE]
+> The default configuration will add 2 sample subscribers. To add a third sample subsciber, edit the configuration variable `NUM_SAMPLE_SUBSCRIBERS` to 3, then re-run `generate.py` and `add-subscribers.py`
 
 3. You can verify if the subscribers were added with
    ```bash
@@ -66,7 +73,6 @@ Ensure your virtual environment is active as described in as described in [Set u
    ```
    2024-10-31 11:55:36 |     INFO | subscriber_1
    2024-10-31 11:55:36 |     INFO | subscriber_2
-   2024-10-31 11:55:36 |     INFO | subscriber_3
    ```
 4.	To delete all subscribers, use delete-subscribers.py.
 ```bash
@@ -76,5 +82,4 @@ Example output:
 ```bash
 2024-10-31 11:56:55 |     INFO | Deleted 001010000000001
 2024-10-31 11:56:55 |     INFO | Deleted 001010000000002
-2024-10-31 11:56:55 |     INFO | Deleted 001010000000003
 ```

data/config.yaml

@@ -5,23 +5,29 @@
 
 yaml = YAML()
 
-DEFAULT_UPLINK_SPEED = {"value": 1, "unit": Open5GS.Unit.Gbps}
-DEFAULT_DOWNLINK_SPEED = {"value": 1, "unit": Open5GS.Unit.Gbps}
-DEFAULT_QOS_INDEX = 9
-DEFAULT_ARP_VALUES = {
-    "priority_level": 8,
-    "pre_emption_capability": Open5GS.Status.DISABLED,
-    "pre_emption_vulnerability": Open5GS.Status.DISABLED,
+######################### CONFIGURATION ###################################
+DEFAULT_CONFIG = {
+    "NUM_SLICES": 2,  # Number of network slices to create
+    "NUM_COTS_SUBSCRIBERS": 0,  # Number of COTS subscribers
+    "NUM_SAMPLE_SUBSCRIBERS": 2,  # Number of simulated subscribers
+    "NUM_AUTO_GENERATED_SUBSCRIBERS": 0,  # Number of auto-generated subscribers (for MSD deployment only)
+    "DEFAULT_UPLINK_SPEED": {"value": 1, "unit": Open5GS.Unit.Gbps},
+    "DEFAULT_DOWNLINK_SPEED": {"value": 1, "unit": Open5GS.Unit.Gbps},
+    "DEFAULT_QOS_INDEX": 9,
+    "DEFAULT_ARP_VALUES": {
+        "priority_level": 8,
+        "pre_emption_capability": Open5GS.Status.DISABLED,
+        "pre_emption_vulnerability": Open5GS.Status.DISABLED,
+    },
+    "DEFAULT_KEY": "465B5CE8B199B49FAA5F0A2EE238A6BC",
+    "DEFAULT_OPC": "E8ED289DEBA952E4283B54E88E6183CA",
+    "DATA_DIR": "data",
+    "SLICE_FILE_PATH": "data/slices.yaml",
+    "SUBSCRIBER_FILE_PATH": "data/subscribers.yaml"
 }
 
-
-DEFAULT_KEY = "465B5CE8B199B49FAA5F0A2EE238A6BC"
-DEFAULT_OPC = "E8ED289DEBA952E4283B54E88E6183CA"
-DATA_DIR = "data"
-
-SLICE_FILE_PATH = f"{DATA_DIR}/slices.yaml"
-SUBSCRIBER_FILE_PATH = f"{DATA_DIR}/subscribers.yaml"
-
+######################### DATA STRUCTURES #################################
+# Example slice and subscriber data for testing.
 slice_data = {
     "slice_1": {
         "sst": 1,
@@ -72,8 +78,7 @@
         ],
     },
 }
-
-subscriber_data = {
+simulated_subscriber_data = {
     "subscriber_1": {
         "_id": "",
         "imsi": "001010000000001",
@@ -138,8 +143,7 @@
         "__v": 0,
     },
 }
-
-cots_ue_data = {
+cots_subscriber_data = {
     "pixel_1": {
         "_id": "",
         "imsi": "001010000060592",
@@ -184,8 +188,33 @@
     },
 }
 
+######################### HELPER FUNCTIONS #################################
+
+def convert_defaultdict_to_dict(obj):
+    """Recursively converts defaultdict to dict."""
+    if isinstance(obj, defaultdict):
+        obj = dict(obj)  # Convert the defaultdict to a regular dict
+    if isinstance(obj, dict):
+        for key, value in obj.items():
+            obj[key] = convert_defaultdict_to_dict(value)  # Recursively convert nested defaultdicts
+    return obj
+
+def load_existing_data():
+    """Load existing slice and subscriber data if available."""
+    global slice_data, simulated_subscriber_data, cots_subscriber_data
+    try:
+        with open(DEFAULT_CONFIG["SLICE_FILE_PATH"], "r") as file:
+            slice_data = yaml.load(file)
+    except FileNotFoundError:
+        log.warning(f"No existing slice data found at {DEFAULT_CONFIG['SLICE_FILE_PATH']}")
 
-def generate_slice_data(slice_number, qos_index):
+    try:
+        with open(DEFAULT_CONFIG["SUBSCRIBER_FILE_PATH"], "r") as file:
+            subscribers = yaml.load(file)
+    except FileNotFoundError:
+        log.warning(f"No existing subscriber data found at {DEFAULT_CONFIG['SUBSCRIBER_FILE_PATH']}")
+
+def generate_slice_data(slice_number, qos_index=DEFAULT_CONFIG["DEFAULT_QOS_INDEX"]):
     slice_name = f"slice_{slice_number}"
     sd = f"{slice_number:06x}"
     session_name = f"dnn{slice_number}"
@@ -201,34 +230,30 @@ def generate_slice_data(slice_number, qos_index):
                     "type": Open5GS.Type.IPv4,
                     "pcc_rule": [],
                     "ambr": {
-                        "uplink": DEFAULT_UPLINK_SPEED,
-                        "downlink": DEFAULT_DOWNLINK_SPEED,
+                        "uplink": DEFAULT_CONFIG["DEFAULT_UPLINK_SPEED"],
+                        "downlink": DEFAULT_CONFIG["DEFAULT_DOWNLINK_SPEED"],
                     },
-                    "qos": {"index": qos_index, "arp": DEFAULT_ARP_VALUES},
+                    "qos": {"index": qos_index, "arp": DEFAULT_CONFIG["DEFAULT_ARP_VALUES"]},
                 }
             ],
         }
     }
 
-
 def create_slices(num_slices):
-    last_slice_number = max(slice_data.keys(), key=lambda x: int(x.split("_")[1]))
+    """Create slices based on the number specified and add to slice_data."""
+    last_slice_number = max(slice_data.keys(), key=lambda x: int(x.split("_")[1]), default="slice_0")
     starting_slice_number = int(last_slice_number.split("_")[1]) + 1
 
     for i in range(starting_slice_number, starting_slice_number + num_slices):
-        qos_index = DEFAULT_QOS_INDEX
-        slice_data.update(generate_slice_data(i, qos_index))
-
+        slice_data.update(generate_slice_data(i))
 
 def generate_subscriber_data(slice_name, subscriber_index):
     slice_info = slice_data[slice_name]
     slice_number = int(slice_name.split("_")[1])
     subscriber_name = f"subscriber_{slice_number}{subscriber_index:02d}"
 
     padding_length = 15 - len(f"00101{slice_name.split('_')[1]}{subscriber_index:02d}")
-    imsi = (
-        f"00101{'0' * padding_length}{slice_name.split('_')[1]}{subscriber_index:02d}"
-    )
+    imsi = f"00101{'0' * padding_length}{slice_name.split('_')[1]}{subscriber_index:02d}"
 
     return {
         subscriber_name: {
@@ -240,65 +265,85 @@ def generate_subscriber_data(slice_name, subscriber_index):
             "access_restriction_data": 32,
             "slice": [slice_info],
             "ambr": {
-                "uplink": DEFAULT_UPLINK_SPEED,
-                "downlink": DEFAULT_DOWNLINK_SPEED,
+                "uplink": DEFAULT_CONFIG["DEFAULT_UPLINK_SPEED"],
+                "downlink": DEFAULT_CONFIG["DEFAULT_DOWNLINK_SPEED"],
             },
             "security": {
-                "k": DEFAULT_KEY,
+                "k": DEFAULT_CONFIG["DEFAULT_KEY"],
                 "amf": "8000",
                 "op": None,
-                "opc": DEFAULT_OPC,
+                "opc": DEFAULT_CONFIG["DEFAULT_OPC"],
             },
             "schema_version": 1,
             "__v": 0,
         }
     }
 
-
-def create_subscribers(num_subscribers):
+def create_auto_generated_subscribers():
+    """Create subscribers and assign them to slices in a round-robin fashion."""
     subscribers = {}
 
-    if config["COTS_UE"]:
-        subscribers.update(cots_ue_data)
+    num_auto_generated_subscribers = DEFAULT_CONFIG["NUM_AUTO_GENERATED_SUBSCRIBERS"]
 
-    if config["SIM_UE"]:
-        subscribers.update(subscriber_data)
-   
+    if num_auto_generated_subscribers > 0:
+        log.info(f"Generating {num_auto_generated_subscribers} auto generated subscribers ...")
 
-    num_slices = len(slice_data)
-    slice_counter = 0
-    subscriber_assignments = defaultdict(list)
-    for _ in range(1, num_subscribers + 1):
-        slice_index = slice_counter % num_slices + 1
-        slice_name = f"slice_{slice_index}"
-        subscriber_index = len(subscriber_assignments[slice_name]) + 1
-        subscriber_assignments[slice_name].append(subscriber_index)
-        slice_counter += 1
-        subscriber = generate_subscriber_data(slice_name, subscriber_index)
-        subscribers.update(subscriber)
+        num_slices = DEFAULT_CONFIG["NUM_SLICES"]
+        slice_counter = 0
+        subscriber_assignments = defaultdict(list)
+        for i in range(1, num_auto_generated_subscribers + 1):
+            slice_index = slice_counter % num_slices + 1
+            slice_name = f"slice_{slice_index}"
+            subscriber_index = len(subscriber_assignments[slice_name]) + 1
+            subscriber_assignments[slice_name].append(subscriber_index)
+            subscriber = generate_subscriber_data(slice_name, subscriber_index)
+            subscribers.update(subscriber)
+            slice_counter += 1
 
     return subscribers
 
+def create_cots_subscribers(subscribers):
+    num_cots_subscribers = DEFAULT_CONFIG["NUM_COTS_SUBSCRIBERS"]
+    if num_cots_subscribers > 0:
+        log.info(f"Adding {num_cots_subscribers} COTS subscribers ...")
+        from itertools import islice
+        for key, value in islice(cots_subscriber_data.items(), num_cots_subscribers):
+            subscribers[key] = value
+
+def create_simulated_subscribers(subscribers):
+    num_simulated_subscribers = DEFAULT_CONFIG["NUM_SAMPLE_SUBSCRIBERS"]
+    if  num_simulated_subscribers > 0:
+        log.info(f"Adding {num_simulated_subscribers} sample subscribers ...")
+        from itertools import islice
+        for key, value in islice(simulated_subscriber_data.items(), num_simulated_subscribers):
+            subscribers[key] = value
+
 
-if __name__ == "__main__":
 
-    with open(DATA_DIR + "/config.yaml", "r") as file:
-        config_file = file.read()
-        config = yaml.load(config_file)
+######################### MAIN SCRIPT ###################################
 
-    num_slices = config["NUM_SLICES"]
-    num_subscribers = config["NUM_SUBSCRIBERS"]
-    existing_slices = len(slice_data)
+def main():
+    log.info("Loading existing data...")
+    load_existing_data()
 
-    log.info(f"Creating {num_slices} slices and {num_subscribers} subscribers")
+    log.info(f"Creating {DEFAULT_CONFIG['NUM_SLICES']} slices ...")
+    slices_to_generate = DEFAULT_CONFIG["NUM_SLICES"] - len(slice_data)
+    log.info(f"Generating {slices_to_generate} new slices...")
+    create_slices(slices_to_generate)
 
-    log.info(f"Creating slices and saving to {SLICE_FILE_PATH}")
-    create_slices(num_slices - existing_slices)
-    with open(SLICE_FILE_PATH, "w") as file:
+    log.info(f"Saving slices to {DEFAULT_CONFIG['SLICE_FILE_PATH']}")
+    with open(DEFAULT_CONFIG["SLICE_FILE_PATH"], "w") as file:
         yaml.dump(slice_data, file)
 
-    log.info(f"Creating subscribers and saving to {SUBSCRIBER_FILE_PATH}")
-    num_subscribers = config["NUM_SUBSCRIBERS"]
-    subscribers = create_subscribers(num_subscribers)
-    with open(SUBSCRIBER_FILE_PATH, "w") as file:
+    subscribers = create_auto_generated_subscribers()
+    create_cots_subscribers(subscribers)
+    create_simulated_subscribers(subscribers)
+
+    log.info(f"Saving subscribers to {DEFAULT_CONFIG['SUBSCRIBER_FILE_PATH']}")
+    with open(DEFAULT_CONFIG["SUBSCRIBER_FILE_PATH"], "w") as file:
         yaml.dump(subscribers, file)
+
+    log.info("Slice and subscriber creation complete.")
+
+if __name__ == "__main__":
+    main()