Skip to content

Commit afc15af

Browse files
hassanctechjdelaplakateyanuragniyatim23disa6302
authored
Merge develop into master (#178)
* Added README instruction to use devleop, and added develop to travis.yml (#135) * Adding a generic utility for exponential backoff retries (#136) * Adding a utility for exponential backoff retries * Adding a utility to compute power Testing: Added unit tests for all relevant changes * updated cmake verison from 2.8 to 3.6.3 (#139) * KVS Events feature (images/notifications) (#138) * Images feature. Tests still incomplete, comments to follow. * Address code review comments, fixed UT, formatted. * Fixed compile errors from enum renames and struct type changes * Unit test floating point issue from %i being able to divide by 0. * Removed variable declaration from for loop and simplified redundant if statements. * Changed enum to be bit shifted ints, allowed to bitwise or multiple events into a single API call * updated struct to include versioning * Style and comment update * removed outdated comment * Define moved back into an enum, macros added for internal bit-wise operations with associated enum * added error messages to default case * spacing for strings tag was incorrect * Additional MKV api tests to ensure the correct formatting of MKV event fragments * Missing declaration in Stream.h for putEventMetadata (#142) * All events will have a unique L1 TAGS. (#145) * Events mkv size bug (#144) * committing to test out changes, added ability to increment the size of tags after appending additional SimpleTags * updating logs * tagsStart initialization was incorrect * cleanup debug logs * Unit test and clang sanitization * accidental leftover comment forward slash * Memory leak in unit test * Removing if else that doesn't cover existing cases * Unnecessary setting of variables that will not be used past this point in the function Co-authored-by: Hassan Sahibzada <[email protected]> * Log git hash * Adding error handler in PIC state machine and updating client/stream state machines to use the error handler on state transitions (#143) * The commit includes following - (1) Refactoring Exponential backoff utility by changing name(s) to use 'RetryStrategy' (2) Minor logging improvements in Exponential backoff utility (3) Adding KVS retry strategy and related callbacks to be used in client and streams (4) Adding a new error handler in PIC state machine (5) Updating KVS client and stream state machines to use PIC error handler and exponential backoff retry strategy on encountering error while state transitions * - Adding client callback to get retry strategy. This is particularly needed for unit testing - Addressing PR feedback * - Updating UTs to handle memory leak for offline streaming - Addressing PR feedback * Addressing PR feedback * Update Client.c trigger travis CI * All events will have a unique L1 TAGS. (#145) * Events mkv size bug (#144) * committing to test out changes, added ability to increment the size of tags after appending additional SimpleTags * updating logs * tagsStart initialization was incorrect * cleanup debug logs * Unit test and clang sanitization * accidental leftover comment forward slash * Memory leak in unit test * Removing if else that doesn't cover existing cases * Unnecessary setting of variables that will not be used past this point in the function Co-authored-by: Hassan Sahibzada <[email protected]> * Log git hash Co-authored-by: Hassan Sahibzada <[email protected]> Co-authored-by: jdelapla <[email protected]> Co-authored-by: Divya Sampath Kumar <[email protected]> * Expose retry count * Move to callback model to get count and move retry state out of public header * Fix potential NULL dereference * Address comments * Move state back to public header since unit tests are failing. Tests will have to fixedi n another PR * Incorporating full jitter implementation of exponential backoff algorithm * Updating stepStateMachine to use stateTransitionHookFunc to get next retry wait time for local state retries (#155) * Updating stepStateMachine to use stateTransitionHookFunc to get next retry wait time for local state retries * Remove check for strategy type (#156) * Test exp for same state * Remove check for strategy type until we handle the STRATEGY_TYPE placement appropriately * Decoupling retry strategy callbacks from the actual retry strategy, adding retry strategy type check before casting retry strategy state * Fix PR feedback and add test for getExponentialBackoffRetryCount * Add missing copy of retry strategy callbacks while creating kinesis video client and fix broken unit test * Crate retry strategy after fixing up device info (#158) * more specific service call results, will be used for clock skew auto correction (#159) * Fix up bad log message (#160) * setup ga; disable travis on develop * fix clang-format check on ga * fix ci file name * Add retry coumt metric * Fix %lu in logging * Trigger CI * Fix format specifiers * fix wiindows, clang, ubsan * fix windows, clang * fix windows, clang * fix clang-format * fix windows * fix windows * fix windows * fix windows * fix windows * fix ubsan and windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build * fix windows build with powershell * fix windows build with powershell * fix windows build with powershell * update call path windows * update call path windows * use aws-actions for creds * add region * fix aws-actions * fix aws-actions * fix aws-actions * use environment * unset session token * unset session token * unset session token * unsetting the token * fix builds * block tags before first cluster, post tags after (#151) * block tags before first cluster, post tags after * missing bool flip * correction to size expectations * use clang 7 * remove continue-on-error * setup arm cross-compilation with qemu * fix msan and ubsan * fix windows build * fix windows build; remove export * fix test instruction * fix test instruction * fix test instruction; get pwd and dir * fix test instruction with backslash * find current directory * check the contents of the build folder * execute the test * disable windows defender * disable windows defender * disable windows defender * fix typo * use docker image for windows * remove msvc action * remove msvc action * use msvc 2017 * use windows 2016 * use vcvars64 * run tests * check verbose logs * update travis and ga for working builds * fix windows build on travis * fix mac builds on ga * update cmake * fix mac builds * fix windows build * initialize firstFrameFirst * fix windows build * fix working builds * move windows build from travis to ga * fix indentation * run tests with arm * fix arm * remove exports, use latest, fix firstFramefirst * set log-level to 2, update mac build name, remove windows from travis * check CC * remove echo * update windows log level * update cross-compilation builds for ga * There are several missing 'accepted' states from the Stream State mac… (#148) * There are several missing 'accepted' states from the Stream State machine * Clang format * Extra accepted states found in Client state machine * I still forget to clang format... * Update StreamState.c trigger travis ci * needed for tagResourceResult Co-authored-by: Hassan Sahibzada <[email protected]> * remove travis.yml * Give Frame a name * Develop (#177) * fix deadlock * fix semaphore release and lock release order * create new mutex around putFrame API call, necessary due to logStreamMetric insidue putframe needing to acquire streamListLock which would otherwise let other threads call putFrame unless we create a new high level lock * set client after we verify it is not NULL Co-authored-by: Hassan Sahibzada <[email protected]> * fix test Co-authored-by: jdelapla <[email protected]> Co-authored-by: Anurag Katey <[email protected]> Co-authored-by: Niyati Maheshwari <[email protected]> Co-authored-by: Divya Sampath Kumar <[email protected]> Co-authored-by: Anurag Katey <[email protected]> Co-authored-by: Dave Johansen <[email protected]> Co-authored-by: Hassan Sahibzada <[email protected]>
1 parent c832588 commit afc15af

File tree

13 files changed

+368
-172
lines changed

13 files changed

+368
-172
lines changed

‎.github/build_windows.bat

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
call "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvars64.bat"
1+
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
22
mkdir build
33
cd build
44
cmake -G "NMake Makefiles" -DBUILD_TEST=TRUE ..

‎.github/workflows/ci.yml

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
name: PIC CI
2+
3+
on:
4+
push:
5+
branches:
6+
- develop
7+
- master
8+
pull_request:
9+
branches:
10+
- develop
11+
- master
12+
jobs:
13+
clang-format-check:
14+
runs-on: macos-latest
15+
steps:
16+
- name: Clone repository
17+
uses: actions/checkout@v2
18+
- name: Install clang-format
19+
run: |
20+
brew install clang-format@8
21+
- name: Run clang format check
22+
run: |
23+
bash scripts/check-clang.sh
24+
mac-os-build-gcc:
25+
runs-on: macos-10.15
26+
env:
27+
CC: gcc-11
28+
CXX: g++-11
29+
AWS_KVS_LOG_LEVEL: 2
30+
steps:
31+
- name: Clone repository
32+
uses: actions/checkout@v2
33+
- name: Build repository
34+
run: |
35+
mkdir build && cd build
36+
sh -c 'cmake .. -DBUILD_TEST=TRUE;cmake .. -DBUILD_TEST=TRUE'
37+
make
38+
./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
39+
mac-os-build-clang:
40+
runs-on: macos-latest
41+
env:
42+
AWS_KVS_LOG_LEVEL: 2
43+
steps:
44+
- name: Clone repository
45+
uses: actions/checkout@v2
46+
- name: Build repository
47+
run: |
48+
mkdir build && cd build
49+
sh -c 'cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE;cmake .. -DBUILD_TEST=TRUE -DCOMPILER_WARNINGS=TRUE'
50+
make
51+
./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
52+
linux-gcc-code-coverage:
53+
runs-on: ubuntu-18.04
54+
env:
55+
AWS_KVS_LOG_LEVEL: 2
56+
steps:
57+
- name: Clone repository
58+
uses: actions/checkout@v2
59+
- name: Build repository
60+
run: |
61+
mkdir build && cd build
62+
cmake .. -DCODE_COVERAGE=TRUE -DBUILD_TEST=TRUE
63+
make
64+
ulimit -c unlimited -S
65+
timeout --signal=SIGABRT 40m ./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
66+
- name: Code coverage
67+
run: |
68+
for test_file in $(find CMakeFiles/kvspic.dir CMakeFiles/kvspicClient.dir CMakeFiles/kvspicState.dir CMakeFiles/kvspicUtils.dir -name '*.gcno'); do gcov $test_file; done
69+
bash <(curl -s https://codecov.io/bash)
70+
address-sanitizer:
71+
runs-on: ubuntu-latest
72+
env:
73+
AWS_KVS_LOG_LEVEL: 2
74+
steps:
75+
- name: Clone repository
76+
uses: actions/checkout@v2
77+
- name: Build repository
78+
run: |
79+
mkdir build && cd build
80+
cmake .. -DBUILD_TEST=TRUE -DADDRESS_SANITIZER=TRUE
81+
make
82+
ulimit -c unlimited -S
83+
timeout --signal=SIGABRT 40m ./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
84+
undefined-behavior-sanitizer:
85+
runs-on: ubuntu-latest
86+
env:
87+
CC: clang-7
88+
CXX: clang++-7
89+
UBSAN_OPTIONS: halt_on_error=1:suppressions=../src/utils/tst/suppressions/UBSAN.supp
90+
AWS_KVS_LOG_LEVEL: 2
91+
steps:
92+
- name: Clone repository
93+
uses: actions/checkout@v2
94+
- name: Install dependencies
95+
run: |
96+
sudo apt-get update
97+
sudo apt-get -y install clang-7
98+
- name: Build repository
99+
run: |
100+
mkdir build && cd build
101+
cmake .. -DBUILD_TEST=TRUE -DUNDEFINED_BEHAVIOR_SANITIZER=TRUE
102+
make
103+
ulimit -c unlimited -S
104+
timeout --signal=SIGABRT 40m ./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
105+
# memory-sanitizer:
106+
# runs-on: ubuntu-18.04
107+
# env:
108+
# CC: clang-7
109+
# CXX: clang++-7
110+
# AWS_KVS_LOG_LEVEL: 2
111+
# steps:
112+
# - name: Clone repository
113+
# uses: actions/checkout@v2
114+
# - name: Install dependencies
115+
# run: |
116+
# sudo apt-get update
117+
# sudo apt-get -y install clang-7
118+
# - name: Build repository
119+
# run: |
120+
# mkdir build && cd build
121+
# cmake .. -DBUILD_TEST=TRUE -DMEMORY_SANITIZER=TRUE
122+
# make
123+
# ulimit -c unlimited -S
124+
# timeout --signal=SIGABRT 40m ./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
125+
# thread-sanitizer:
126+
# runs-on: ubuntu-18.04
127+
# env:
128+
# CC: clang-7
129+
# CXX: clang++-7
130+
# AWS_KVS_LOG_LEVEL: 2
131+
# TSAN_OPTIONS: halt_on_error=1:suppressions=../src/utils/tst/suppressions/TSAN.supp
132+
# steps:
133+
# - name: Clone repository
134+
# uses: actions/checkout@v2
135+
# - name: Install dependencies
136+
# run: |
137+
# sudo apt-get update
138+
# sudo apt-get -y install clang-7
139+
# - name: Build repository
140+
# run: |
141+
# mkdir build && cd build
142+
# cmake .. -DBUILD_TEST=TRUE -DTHREAD_SANITIZER=TRUE
143+
# make
144+
# ./kvspic_test --gtest_break_on_failure --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*:ThreadFunctionalityTest.ThreadCreateAndCancel"
145+
linux-gcc-4_4:
146+
runs-on: ubuntu-18.04
147+
env:
148+
AWS_KVS_LOG_LEVEL: 2
149+
CC: gcc-4.4
150+
steps:
151+
- name: Clone repository
152+
uses: actions/checkout@v2
153+
- name: Install deps
154+
run: |
155+
sudo sh -c 'echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6'
156+
sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
157+
sudo add-apt-repository 'deb http://archive.ubuntu.com/ubuntu/ trusty main'
158+
sudo add-apt-repository 'deb http://archive.ubuntu.com/ubuntu/ trusty universe'
159+
sudo apt-get -q update
160+
sudo apt-get -y install gcc-4.4
161+
sudo apt-get -y install gdb
162+
- name: Build repository
163+
run: |
164+
mkdir build && cd build
165+
cmake .. -DBUILD_TEST=TRUE
166+
make
167+
ulimit -c unlimited -S
168+
timeout --signal=SIGABRT 40m ./kvspic_test --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*"
169+
windows-msvc:
170+
runs-on: windows-2022
171+
env:
172+
AWS_KVS_LOG_LEVEL: 7
173+
steps:
174+
- name: Clone repository
175+
uses: actions/checkout@v2
176+
- name: Build and run
177+
run: |
178+
.github/build_windows.bat
179+
D:\a\amazon-kinesis-video-streams-pic\amazon-kinesis-video-streams-pic\build\kvspic_test.exe --gtest_filter="-TimerQueueFunctionalityTest.*:HeapPerfTest.*:PermutatedStreamInfo/StateTransitionFunctionalityTest.ControlPlaneServiceCallExhaustRetry*:PermutatedStreamInfo/IntermittentProducerAutomaticStreamingTest.ValidateTimerInvokedBeforeTime*:PermutatedStreamInfo/IntermittentProducerAutomaticStreamingTest.ValidateTimerInvokedAfterFirstPeriod*:PermutatedStreamInfo/IntermittentProducerAutomaticStreamingTest.ValidateLastUpdateTimeOfStreamUpdated*:PermutatedStreamInfo/IntermittentProducerAutomaticStreamingTest.MultiTrackVerifyNoInvocationsWithSingleTrackProducer*:PermutatedStreamInfo/IntermittentProducerAutomaticStreamingTest.ValidateNoConsecutiveEOFR*:PermutatedStreamInfo/IntermittentProducerAutomaticStreamingTest.ValidateErrorOnForceConsecutiveEOFR*:*StreamStateTransitionsTest*:*PermutatedStreamInfo/StateTransitionFunctionalityTest.StreamTerminatedAndGoToGetEndpointState*:*PermutatedStreamInfo/StateTransitionFunctionalityTest.StreamTerminatedAndGoToDescribeState*:*PermutatedStreamInfo/StateTransitionFunctionalityTest*"
180+
arm64-cross-compilation:
181+
runs-on: ubuntu-latest
182+
env:
183+
CC: aarch64-linux-gnu-gcc
184+
CXX: aarch64-linux-gnu-g++
185+
steps:
186+
- name: Install dependencies
187+
run: |
188+
sudo apt update
189+
sudo apt-get -y install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu binutils-aarch64-linux-gnu
190+
- name: Clone repository
191+
uses: actions/checkout@v2
192+
- name: Build Repository
193+
run: |
194+
sudo sh -c 'echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6'
195+
mkdir build && cd build
196+
cmake .. -DBUILD_TEST=TRUE
197+
make
198+
arm32-cross-compilation:
199+
runs-on: ubuntu-latest
200+
env:
201+
CC: arm-linux-gnueabi-gcc
202+
CXX: arm-linux-gnueabi-g++
203+
steps:
204+
- name: Install dependencies
205+
run: |
206+
sudo apt update
207+
sudo apt-get -y install gcc-arm-linux-gnueabi g++-arm-linux-gnueabi binutils-arm-linux-gnueabi
208+
- name: Clone repository
209+
uses: actions/checkout@v2
210+
- name: Build Repository
211+
run: |
212+
sudo sh -c 'echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6'
213+
mkdir build && cd build
214+
cmake .. -DBUILD_TEST=TRUE
215+
make
216+

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,4 @@ cmake-build-debug/
66
cmake-build-release/
77
open-source/
88
outputs
9+
cmake-build-pic/

‎.travis.yml

Lines changed: 0 additions & 143 deletions
This file was deleted.

0 commit comments

Comments
 (0)