Skip to content

Give sufficient time to get inference statistics for sequence tests #5670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 21, 2023
Merged

Give sufficient time to get inference statistics for sequence tests #5670

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8872e85
If statistics not ready, wait up to 10s to check them
dyastremsky Apr 20, 2023
b295c1e
Add debug message to see if wait helps avoid test failure
dyastremsky Apr 20, 2023
ce8ba48
Comment timing logic, print loop, add verbose logs
dyastremsky Apr 20, 2023
37ccc35
Use tries as loop iterator
dyastremsky Apr 20, 2023
8792af7
Change back to 10 seconds
dyastremsky Apr 21, 2023
be0ecb3
Make 10 seconds by extending sleep
dyastremsky Apr 21, 2023
8c7e6da
Remove unused variable start_time
dyastremsky Apr 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion qa/L0_sequence_batcher/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ else
fi
fi

SERVER_ARGS_EXTRA="--backend-directory=${BACKEND_DIR} --backend-config=tensorflow,version=${TF_VERSION}"
SERVER_ARGS_EXTRA="--backend-directory=${BACKEND_DIR} --backend-config=tensorflow,version=${TF_VERSION} --log-verbose=1"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating verbose logging so that we have more information if this fails again. SERVER_ARGS_EXTRA is used every time, so putting this here seemed to make more sense than putting it in five places.


source ../common/util.sh

Expand Down
17 changes: 15 additions & 2 deletions qa/common/sequence_util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -984,8 +984,21 @@ def check_setup(self, model_name):
_max_sequence_idle_ms * 1000) # 5 secs

def check_status(self, model_name, batch_exec, exec_cnt, infer_cnt):
stats = self.triton_client_.get_inference_statistics(model_name, "1")
self.assertEqual(len(stats.model_stats), 1, "expect 1 model stats")
# There is a time window between when responses are returned and statistics are updated.
# To prevent intermittent test failure during that window, wait up to 10 seconds for the
# inference statistics to be ready.
num_tries = 10
for i in range(num_tries):
stats = self.triton_client_.get_inference_statistics(
model_name, "1")
self.assertEqual(len(stats.model_stats), 1, "expect 1 model stats")
actual_exec_cnt = stats.model_stats[0].execution_count
if actual_exec_cnt == exec_cnt:
break
print("WARNING: expect {} executions, got {} (attempt {})".format(
exec_cnt, actual_exec_cnt, i))
time.sleep(1)

self.assertEqual(stats.model_stats[0].name, model_name,
"expect model stats for model {}".format(model_name))
self.assertEqual(
Expand Down