Skip to content

Commit 38c6b89

Browse files
doctorlai-msrcdoctorlai-msrc
authored
Ensure load after unload works (#67)
* Add test load&unload twice * Revert * Debug * Fix warnings * Remove unstable assert * Cleanup * Manual workflow trigger * Revert * Revert * Cleanup * Fix * Cleanup * Remove misleading message * Temp disable _py when python version is 3.10.x * Test Python 3.12 on ubuntu22.04 * Fix * Debug * Revert debugging * Debug * Revert * lock the gil when creating interpreter * Remove lock * Revert dockerfile in ubuntu22.04 * Remove ubuntu22.04 support
1 parent f14a98c commit 38c6b89

File tree

21 files changed

+123
-155
lines changed

21 files changed

+123
-155
lines changed

‎.github/workflows/docker-build-and-test.yaml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ on:
1919
type: choice
2020
options:
2121
- azurelinux
22-
- ubuntu22_04
2322
- ubuntu24_04
2423
testCase:
2524
description: 'Test Case'
@@ -47,7 +46,6 @@ jobs:
4746
matrix:
4847
dockerType:
4948
- { value: "azurelinux", displayName: "Azure Linux", id: "mariner" }
50-
- { value: "ubuntu22_04", displayName: "Ubuntu 22.04", id: "ubuntu22_04" }
5149
- { value: "ubuntu24_04", displayName: "Ubuntu 24.04", id: "ubuntu24_04" }
5250
testCase:
5351
- { description: "Address Sanitizer", id: "test1", sanitizerBuildParam: "-e SANITIZER=1" }

‎deploy/ubuntu22_04.Dockerfile

Lines changed: 0 additions & 60 deletions
This file was deleted.

‎docs/containers.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,17 @@ We provide standard Dockerfiles to build *jrt-controller* images for standard op
77
The Dockerfiles in this directory can be split into two categories:
88
1. builder images for different operating systems:
99
* [azurelinux](../deploy/azurelinux.Dockerfile)
10-
* [Ubuntu22.04](../deploy/ubuntu22_04.Dockerfile)
1110
* [Ubuntu24.04](../deploy/ubuntu24_04.Dockerfile)
1211
2. Containerized application
1312
* [jrtc-ctl](../deploy/jrtc-ctl.Dockerfile) is built from one of the OS images in step 1.
1413

14+
Please note that we currently don't support Ubuntu 22.04 as the JRTC Python App is not compatible with it.
15+
1516
## Build
1617

1718
```sh
1819
# To build for a particular OS, run:
19-
OS=azurelinux # possible values: [ubuntu22_04, azurelinux, ubuntu24_04]
20+
OS=azurelinux # possible values: [azurelinux, ubuntu24_04]
2021

2122
# Create builder image with all dependencies loaded
2223
sudo -E docker build -t jrtc-$OS:latest -f deploy/$OS.Dockerfile .

‎helper_build_files/run_all_integration_tests.sh

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ if ! pushd "$JRTC_PATH/helper_build_files"; then
1313
fi
1414

1515
TEST_CASES=("advanced_example_py" "first_example_py" "first_example" "first_example_c" "advanced_example" "advanced_example_c")
16+
echo "Running integration tests for the following cases: ${TEST_CASES[*]}"
17+
1618
JRTC_TESTS_OUTPUT=/tmp/jrtc_tests_output.log
1719

1820
for TEST in "${TEST_CASES[@]}"; do

‎pipeline/doxygen.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,10 @@ steps:
66
- template: print-debug-info.yaml
77
- template: build-lib.yaml
88
parameters:
9-
dockerType: ubuntu22_04
9+
dockerType: ubuntu24_04
1010
- bash: |
1111
set -x
12-
if ! docker run -v `pwd`:/jrtc_out_lib -e DOXYGEN=1 $(containerRegistry)/jrtc-ubuntu22_04:$(imageTag); then
12+
if ! docker run -v `pwd`:/jrtc_out_lib -e DOXYGEN=1 $(containerRegistry)/jrtc-ubuntu24_04:$(imageTag); then
1313
echo ERROR make doc
1414
exit 1
1515
fi

‎sample_apps/advanced_example/Readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ value to the simple_input codelet via a control_input channel. The simple_input
2323
This example assumes that you already have Jbpf and you have already built the Jbpf library without DPDK support (cmake option `-DUSE_DPDK=off`).
2424
Also, it assumes that you are running all the below steps as root.
2525

26-
Steps to run the example. The followinf examples use "ubuntu2204" as the OS, but "mariner" could be used too.
26+
Steps to run the example.
2727

2828
# 2. Steps
2929

‎sample_apps/advanced_example_c/Readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ value to the simple_input codelet via a control_input channel. The simple_input
2323
This example assumes that you already have Jbpf and you have already built the Jbpf library without DPDK support (cmake option `-DUSE_DPDK=off`).
2424
Also, it assumes that you are running all the below steps as root.
2525

26-
Steps to run the example. The followinf examples use "ubuntu2204" as the OS, but "mariner" could be used too.
26+
Steps to run the example.
2727

2828
# 2. Steps
2929

‎sample_apps/advanced_example_py/Readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ value to the simple_input codelet via a control_input channel. The simple_input
2323
This example assumes that you already have Jbpf and you have already built the Jbpf library without DPDK support (cmake option `-DUSE_DPDK=off`).
2424
Also, it assumes that you are running all the below steps as root.
2525

26-
Steps to run the example. The followinf examples use "ubuntu2204" as the OS, but "mariner" could be used too.
26+
Steps to run the example.
2727

2828
# 2. Steps
2929

‎sample_apps/advanced_example_py/assert.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,4 +34,11 @@ if [[ $count -ne 0 ]]; then
3434
exit 1
3535
fi
3636

37+
### This should appear twice: App2: Aggregate counter so far is 15
38+
count=$(grep -c "App2: Aggregate counter so far is 15" $1)
39+
if [[ $count -ne 2 ]]; then
40+
echo "Expected 'App2: Aggregate counter so far is 15' to appear twice, but found $count times."
41+
exit 1
42+
fi
43+
3744
exit 0

‎sample_apps/advanced_example_py/run.sh

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,4 +89,19 @@ sleep 20
8989
echo "Running: Unload YAML"
9090
cd $JRTC_PATH/sample_apps/advanced_example_py && source ../../setup_jrtc_env.sh && ./unload_app.sh
9191

92+
echo "---------------------- Now load and unload to make sure if all works again ----------------------"
93+
94+
# Step 7: Load YAML (runs in the foreground)
95+
echo "Running: Load YAML"
96+
cd $JRTC_PATH/sample_apps/advanced_example_py && source ../../setup_jrtc_env.sh && ./load_app.sh
97+
98+
## Wait to see output
99+
echo "Waiting for output..."
100+
sleep 20
101+
102+
# Step 8: Unload YAML (runs in the foreground)
103+
echo "Running: Unload YAML"
104+
cd $JRTC_PATH/sample_apps/advanced_example_py && source ../../setup_jrtc_env.sh && ./unload_app.sh
105+
106+
92107
echo "Test completed."

‎sample_apps/first_example/Readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ value to the simple_input codelet via a control_input channel. The simple_input
99
This example assumes that you already have Jbpf and you have already built the Jbpf library without DPDK support (cmake option `-DUSE_DPDK=off`).
1010
Also, it assumes that you are running all the below steps as root.
1111

12-
Steps to run the example. The following examples use "ubuntu2204" as the OS, but "mariner" could be used too.
12+
Steps to run the example.
1313

1414
## Usage
1515

‎sample_apps/first_example_c/Readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ value to the simple_input codelet via a control_input channel. The simple_input
99
This example assumes that you already have Jbpf and you have already built the Jbpf library without DPDK support (cmake option `-DUSE_DPDK=off`).
1010
Also, it assumes that you are running all the below steps as root.
1111

12-
Steps to run the example. The following examples use "ubuntu2204" as the OS, but "mariner" could be used too.
12+
Steps to run the example.
1313

1414
## Usage
1515

‎sample_apps/first_example_py/Readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ value to the simple_input codelet via a control_input channel. The simple_input
2525
This example assumes that you already have Jbpf and you have already built the Jbpf library without DPDK support (cmake option `-DUSE_DPDK=off`).
2626
Also, it assumes that you are running all the below steps as root.
2727

28-
Steps to run the example. The followinf examples use "ubuntu2204" as the OS, but "mariner" could be used too.
28+
Steps to run the example.
2929

3030
# 2. Steps
3131

‎sample_apps/first_example_py/assert.sh

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,10 @@ if [[ $count -ne 0 ]]; then
2828
exit 1
2929
fi
3030

31+
### This should appear twice: Aggregate counter so far is: 15
32+
count=$(grep -c "Aggregate counter so far is: 15" $1)
33+
if [[ $count -ne 2 ]]; then
34+
echo "Expected 'Aggregate counter so far is: 15' to appear twice, but found $count times."
35+
exit 1
36+
fi
3137
exit 0

‎sample_apps/first_example_py/first_example.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@ def app_handler(
7474
##########################################################################
7575
# Main function to start the app (converted from jrtc_start_app)
7676
def jrtc_start_app(capsule):
77+
print("Starting FirstExample app...", flush=True)
78+
7779
streams = [
7880
# GENERATOR_OUT_STREAM_IDX
7981
JrtcStreamCfg_t(

‎sample_apps/first_example_py/run.sh

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,28 @@ sleep 20
9393
echo "Running: Unload YAML"
9494
cd $JRTC_PATH/sample_apps/first_example_py && source ../../setup_jrtc_env.sh
9595

96-
## to test if we can unload in any directory
96+
cd $JRTC_PATH
97+
sample_apps/first_example_py/unload_app.sh
98+
99+
echo "---------------------- Now load and unload to make sure if all works again ----------------------"
100+
sleep 5
101+
102+
### Step 7: Load YAML (runs in the foreground)
103+
echo "Running: Load YAML"
104+
cd $JRTC_PATH/sample_apps/first_example_py && source ../../setup_jrtc_env.sh
105+
106+
cd $JRTC_PATH
107+
sample_apps/first_example_py/load_app.sh
108+
109+
## Wait to see output
110+
echo "Waiting for output..."
111+
sleep 20
112+
113+
# Step 8: Unload YAML (runs in the foreground)
114+
echo "Running: Unload YAML"
115+
cd $JRTC_PATH/sample_apps/first_example_py && source ../../setup_jrtc_env.sh
116+
117+
## Unload again
97118
cd $JRTC_PATH
98119
sample_apps/first_example_py/unload_app.sh
99120

‎src/controller/jrtc_int.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,8 @@
3535
// Global shared Python state, only one instance
3636
shared_python_state_t shared_python_state = {
3737
.python_lock = PTHREAD_MUTEX_INITIALIZER,
38-
.active_interpreter_users = ATOMIC_VAR_INIT(0),
39-
.python_initialized = ATOMIC_VAR_INIT(0),
38+
.main_ts = NULL,
39+
.initialized = false,
4040
};
4141

4242
/* Compiler magic to make address sanitizer ignore

‎src/controller/jrtc_shared_python_state.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@
77
typedef struct shared_python_state
88
{
99
pthread_mutex_t python_lock;
10-
atomic_int active_interpreter_users;
11-
atomic_int python_initialized;
10+
PyThreadState* main_ts; // Store the main interpreter thread state
11+
bool initialized;
1212
} shared_python_state_t;
1313

1414
#endif

0 commit comments

Comments
 (0)