[source-amazon-ads] - sponsored_products_report_stream - No Rows Being Acquired - Logs Show 400 HTTP Code Error

[SalvitSimmus7]

Connector Name

amazon-ads

Connector Version

5.0.9

What step the error happened?

During the sync

Relevant information

I am encountering an issue with the Amazon Ads source, specifically with the "sponsored_products_report_stream". Despite expecting to receive data because I know it exists in the profile, the stream is returning no rows every time I sync. The logs indicate numerous HTTP 400 status codes, as if there was a problem with the request sent to the Amazon Ads API.

However, if I go to the links of the logs after some time manually, I am able to download the report data. It seems like if Airbyte connection times out too fast, not being able to get the report's data. I wonder if this issue can be fixed adding more timeout so the backoff mechanism is able to download a bit later after 5 tries

logs.txt

Relevant log output

2024-07-18 15:14:52 source > Checking report status, 7 report(s) remaining
2024-07-18 15:14:53 source > Backing off _download_report(...) for 0.2s (requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://offline-report-storage-us-east-1-prod.s3.amazonaws.com/0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029/report-0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029.json.gz?X-Amz-Security-Token=IQoJb3JpZ2luX2VjED8aCXVzLWVhc3QtMSJIMEYCIQCBaeaTnmm%2F%2FIt3q3Am0mNzyxrRzolA0Y2fhvgM3LZESAIhANKTjrs2Yt9PXJc9%2FuAkfTjSppk58FZjCY3NoV0xdznkKuEFCBcQAhoMODQxNjQ1NTQzMDc0IgziqpiXotr%2FVafuD4gqvgUcDuKSrJlwtdXNrs8YZVUxDarVypQiwfi3MWMXgkedEWXcIOTAtBBUNoU8QFuZMIGymN4wyMRpgyhljZUJfqCozBj1KM9DU8hp5Pc9s9EBpJ%2BO3yf8eqO03%2FR9aUnyHGfY2QA39cor952%2FOBB1VUCVbiY4ihdxveuVGSeNiOZzN1OVhip6%2B%2B2fXdJ6aA4XL7y49q6KJMj45EepWHfXifHmBDUI%2F3AD1d%2B3%2BDyFWBzqEqK0YbNXhgL3sm2j8KsmFY%2FjSq7cmRMzmBdTTQn%2FwKz05z4Nc6t6PwM8gamJh5vDCDYRQHTPY2NletI6oQZQYyLeRfpAHSoLBFlqgFIas%2FRHORXFl3ZIP159SPbbsS9kd0NYgDYlgAQOVZat%2FBgCuvGLdYJLO9ZDW00x13pYga7lZ%2BGi0LG0qNm5%2Fzfyhkv42Di%2BGnaDBL%2B0CVCWPtAKKoW2qSHCRuhPWU%2B0R2Pq6dW1AQ00J7SSDpWYi%2Ff%2BWWMT4dE%2FU00jS6TeHc7CvcK5ma%2FBBwJeF0RvKDlUgE8wBCK4NDP0cOsN5qkdlvjnptU4ACz2G2FECnYbbwFs%2Fh7q3z3j%2BT4TFT5OQBHmMuEUq2Ez%2FEc%2FQcHqyPkz0NLXDSoil%2FYHXpcRO%2FpF3AT5dCvK1k8ME%2BByuLtwMv8Ysx6EiCIRUzt%2FCAtFmTYHMn0vtIuZJ3RzItYQ%2Fn9AP3WS%2FGvy48PaHbSMRE1aqCocrF294NtSMPH8pZn2fgZN8TL82iqNiCVmjNusOwDXIyemWJqwjURnrpGST01Uz%2F8EXa51VfJ%2Bq6%2FWB89zgIfdtsKFBdT50PYPlPGW3RnVIXtNqq0y17Yd%2FWcQjU8KwoG79jPHOD1VmNTYbfMAAmj5pmk98rVpkQCjgShkLMdgXqpR6xuVuU05FIZ9iB%2Bwmtjc7PkD9iin9BIz3lys3ieJ8lUbP%2Bww4MjktAY6qAE5thsN02OXfxiuraGEy3kzmgki8HqDlO5maGS3I3pa6ZLQcYRVpdSo4XBbQnyh8J1IwrUmcK3YPwXMZBLfAsMRECB12KKFwjH8hRQdcs2djqBqKT63kTrymZ21SpzOxr8TPw1dSl3GEpVNWQ9xtuttzS2kkmDT6MfqBZGF2oBOMO1gnJ%2BfcalZdgsDqiCVH7cm%2Fx0fvwPe6iuxCHbefyW9HDyc1B84xzA%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240718T151453Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=ASIA4H5P3Z2RCNHWJUDF%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=f9c4198451d382120ba9517c5644a836940ab4de5708b5787c7fe8d2da9f4475)
2024-07-18 15:14:53 source > Backing off _download_report(...) for 0.5s (requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://offline-report-storage-us-east-1-prod.s3.amazonaws.com/0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029/report-0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029.json.gz?X-Amz-Security-Token=IQoJb3JpZ2luX2VjED8aCXVzLWVhc3QtMSJIMEYCIQCBaeaTnmm%2F%2FIt3q3Am0mNzyxrRzolA0Y2fhvgM3LZESAIhANKTjrs2Yt9PXJc9%2FuAkfTjSppk58FZjCY3NoV0xdznkKuEFCBcQAhoMODQxNjQ1NTQzMDc0IgziqpiXotr%2FVafuD4gqvgUcDuKSrJlwtdXNrs8YZVUxDarVypQiwfi3MWMXgkedEWXcIOTAtBBUNoU8QFuZMIGymN4wyMRpgyhljZUJfqCozBj1KM9DU8hp5Pc9s9EBpJ%2BO3yf8eqO03%2FR9aUnyHGfY2QA39cor952%2FOBB1VUCVbiY4ihdxveuVGSeNiOZzN1OVhip6%2B%2B2fXdJ6aA4XL7y49q6KJMj45EepWHfXifHmBDUI%2F3AD1d%2B3%2BDyFWBzqEqK0YbNXhgL3sm2j8KsmFY%2FjSq7cmRMzmBdTTQn%2FwKz05z4Nc6t6PwM8gamJh5vDCDYRQHTPY2NletI6oQZQYyLeRfpAHSoLBFlqgFIas%2FRHORXFl3ZIP159SPbbsS9kd0NYgDYlgAQOVZat%2FBgCuvGLdYJLO9ZDW00x13pYga7lZ%2BGi0LG0qNm5%2Fzfyhkv42Di%2BGnaDBL%2B0CVCWPtAKKoW2qSHCRuhPWU%2B0R2Pq6dW1AQ00J7SSDpWYi%2Ff%2BWWMT4dE%2FU00jS6TeHc7CvcK5ma%2FBBwJeF0RvKDlUgE8wBCK4NDP0cOsN5qkdlvjnptU4ACz2G2FECnYbbwFs%2Fh7q3z3j%2BT4TFT5OQBHmMuEUq2Ez%2FEc%2FQcHqyPkz0NLXDSoil%2FYHXpcRO%2FpF3AT5dCvK1k8ME%2BByuLtwMv8Ysx6EiCIRUzt%2FCAtFmTYHMn0vtIuZJ3RzItYQ%2Fn9AP3WS%2FGvy48PaHbSMRE1aqCocrF294NtSMPH8pZn2fgZN8TL82iqNiCVmjNusOwDXIyemWJqwjURnrpGST01Uz%2F8EXa51VfJ%2Bq6%2FWB89zgIfdtsKFBdT50PYPlPGW3RnVIXtNqq0y17Yd%2FWcQjU8KwoG79jPHOD1VmNTYbfMAAmj5pmk98rVpkQCjgShkLMdgXqpR6xuVuU05FIZ9iB%2Bwmtjc7PkD9iin9BIz3lys3ieJ8lUbP%2Bww4MjktAY6qAE5thsN02OXfxiuraGEy3kzmgki8HqDlO5maGS3I3pa6ZLQcYRVpdSo4XBbQnyh8J1IwrUmcK3YPwXMZBLfAsMRECB12KKFwjH8hRQdcs2djqBqKT63kTrymZ21SpzOxr8TPw1dSl3GEpVNWQ9xtuttzS2kkmDT6MfqBZGF2oBOMO1gnJ%2BfcalZdgsDqiCVH7cm%2Fx0fvwPe6iuxCHbefyW9HDyc1B84xzA%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240718T151453Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=ASIA4H5P3Z2RCNHWJUDF%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=f9c4198451d382120ba9517c5644a836940ab4de5708b5787c7fe8d2da9f4475)
2024-07-18 15:14:54 source > Backing off _download_report(...) for 2.6s (requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://offline-report-storage-us-east-1-prod.s3.amazonaws.com/0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029/report-0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029.json.gz?X-Amz-Security-Token=IQoJb3JpZ2luX2VjED8aCXVzLWVhc3QtMSJIMEYCIQCBaeaTnmm%2F%2FIt3q3Am0mNzyxrRzolA0Y2fhvgM3LZESAIhANKTjrs2Yt9PXJc9%2FuAkfTjSppk58FZjCY3NoV0xdznkKuEFCBcQAhoMODQxNjQ1NTQzMDc0IgziqpiXotr%2FVafuD4gqvgUcDuKSrJlwtdXNrs8YZVUxDarVypQiwfi3MWMXgkedEWXcIOTAtBBUNoU8QFuZMIGymN4wyMRpgyhljZUJfqCozBj1KM9DU8hp5Pc9s9EBpJ%2BO3yf8eqO03%2FR9aUnyHGfY2QA39cor952%2FOBB1VUCVbiY4ihdxveuVGSeNiOZzN1OVhip6%2B%2B2fXdJ6aA4XL7y49q6KJMj45EepWHfXifHmBDUI%2F3AD1d%2B3%2BDyFWBzqEqK0YbNXhgL3sm2j8KsmFY%2FjSq7cmRMzmBdTTQn%2FwKz05z4Nc6t6PwM8gamJh5vDCDYRQHTPY2NletI6oQZQYyLeRfpAHSoLBFlqgFIas%2FRHORXFl3ZIP159SPbbsS9kd0NYgDYlgAQOVZat%2FBgCuvGLdYJLO9ZDW00x13pYga7lZ%2BGi0LG0qNm5%2Fzfyhkv42Di%2BGnaDBL%2B0CVCWPtAKKoW2qSHCRuhPWU%2B0R2Pq6dW1AQ00J7SSDpWYi%2Ff%2BWWMT4dE%2FU00jS6TeHc7CvcK5ma%2FBBwJeF0RvKDlUgE8wBCK4NDP0cOsN5qkdlvjnptU4ACz2G2FECnYbbwFs%2Fh7q3z3j%2BT4TFT5OQBHmMuEUq2Ez%2FEc%2FQcHqyPkz0NLXDSoil%2FYHXpcRO%2FpF3AT5dCvK1k8ME%2BByuLtwMv8Ysx6EiCIRUzt%2FCAtFmTYHMn0vtIuZJ3RzItYQ%2Fn9AP3WS%2FGvy48PaHbSMRE1aqCocrF294NtSMPH8pZn2fgZN8TL82iqNiCVmjNusOwDXIyemWJqwjURnrpGST01Uz%2F8EXa51VfJ%2Bq6%2FWB89zgIfdtsKFBdT50PYPlPGW3RnVIXtNqq0y17Yd%2FWcQjU8KwoG79jPHOD1VmNTYbfMAAmj5pmk98rVpkQCjgShkLMdgXqpR6xuVuU05FIZ9iB%2Bwmtjc7PkD9iin9BIz3lys3ieJ8lUbP%2Bww4MjktAY6qAE5thsN02OXfxiuraGEy3kzmgki8HqDlO5maGS3I3pa6ZLQcYRVpdSo4XBbQnyh8J1IwrUmcK3YPwXMZBLfAsMRECB12KKFwjH8hRQdcs2djqBqKT63kTrymZ21SpzOxr8TPw1dSl3GEpVNWQ9xtuttzS2kkmDT6MfqBZGF2oBOMO1gnJ%2BfcalZdgsDqiCVH7cm%2Fx0fvwPe6iuxCHbefyW9HDyc1B84xzA%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240718T151453Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=ASIA4H5P3Z2RCNHWJUDF%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=f9c4198451d382120ba9517c5644a836940ab4de5708b5787c7fe8d2da9f4475)
2024-07-18 15:14:57 source > Backing off _download_report(...) for 6.4s (requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://offline-report-storage-us-east-1-prod.s3.amazonaws.com/0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029/report-0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029.json.gz?X-Amz-Security-Token=IQoJb3JpZ2luX2VjED8aCXVzLWVhc3QtMSJIMEYCIQCBaeaTnmm%2F%2FIt3q3Am0mNzyxrRzolA0Y2fhvgM3LZESAIhANKTjrs2Yt9PXJc9%2FuAkfTjSppk58FZjCY3NoV0xdznkKuEFCBcQAhoMODQxNjQ1NTQzMDc0IgziqpiXotr%2FVafuD4gqvgUcDuKSrJlwtdXNrs8YZVUxDarVypQiwfi3MWMXgkedEWXcIOTAtBBUNoU8QFuZMIGymN4wyMRpgyhljZUJfqCozBj1KM9DU8hp5Pc9s9EBpJ%2BO3yf8eqO03%2FR9aUnyHGfY2QA39cor952%2FOBB1VUCVbiY4ihdxveuVGSeNiOZzN1OVhip6%2B%2B2fXdJ6aA4XL7y49q6KJMj45EepWHfXifHmBDUI%2F3AD1d%2B3%2BDyFWBzqEqK0YbNXhgL3sm2j8KsmFY%2FjSq7cmRMzmBdTTQn%2FwKz05z4Nc6t6PwM8gamJh5vDCDYRQHTPY2NletI6oQZQYyLeRfpAHSoLBFlqgFIas%2FRHORXFl3ZIP159SPbbsS9kd0NYgDYlgAQOVZat%2FBgCuvGLdYJLO9ZDW00x13pYga7lZ%2BGi0LG0qNm5%2Fzfyhkv42Di%2BGnaDBL%2B0CVCWPtAKKoW2qSHCRuhPWU%2B0R2Pq6dW1AQ00J7SSDpWYi%2Ff%2BWWMT4dE%2FU00jS6TeHc7CvcK5ma%2FBBwJeF0RvKDlUgE8wBCK4NDP0cOsN5qkdlvjnptU4ACz2G2FECnYbbwFs%2Fh7q3z3j%2BT4TFT5OQBHmMuEUq2Ez%2FEc%2FQcHqyPkz0NLXDSoil%2FYHXpcRO%2FpF3AT5dCvK1k8ME%2BByuLtwMv8Ysx6EiCIRUzt%2FCAtFmTYHMn0vtIuZJ3RzItYQ%2Fn9AP3WS%2FGvy48PaHbSMRE1aqCocrF294NtSMPH8pZn2fgZN8TL82iqNiCVmjNusOwDXIyemWJqwjURnrpGST01Uz%2F8EXa51VfJ%2Bq6%2FWB89zgIfdtsKFBdT50PYPlPGW3RnVIXtNqq0y17Yd%2FWcQjU8KwoG79jPHOD1VmNTYbfMAAmj5pmk98rVpkQCjgShkLMdgXqpR6xuVuU05FIZ9iB%2Bwmtjc7PkD9iin9BIz3lys3ieJ8lUbP%2Bww4MjktAY6qAE5thsN02OXfxiuraGEy3kzmgki8HqDlO5maGS3I3pa6ZLQcYRVpdSo4XBbQnyh8J1IwrUmcK3YPwXMZBLfAsMRECB12KKFwjH8hRQdcs2djqBqKT63kTrymZ21SpzOxr8TPw1dSl3GEpVNWQ9xtuttzS2kkmDT6MfqBZGF2oBOMO1gnJ%2BfcalZdgsDqiCVH7cm%2Fx0fvwPe6iuxCHbefyW9HDyc1B84xzA%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240718T151453Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=ASIA4H5P3Z2RCNHWJUDF%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=f9c4198451d382120ba9517c5644a836940ab4de5708b5787c7fe8d2da9f4475)
2024-07-18 15:15:04 source > Giving up _download_report(...) after 5 tries (requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://offline-report-storage-us-east-1-prod.s3.amazonaws.com/0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029/report-0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029.json.gz?X-Amz-Security-Token=IQoJb3JpZ2luX2VjED8aCXVzLWVhc3QtMSJIMEYCIQCBaeaTnmm%2F%2FIt3q3Am0mNzyxrRzolA0Y2fhvgM3LZESAIhANKTjrs2Yt9PXJc9%2FuAkfTjSppk58FZjCY3NoV0xdznkKuEFCBcQAhoMODQxNjQ1NTQzMDc0IgziqpiXotr%2FVafuD4gqvgUcDuKSrJlwtdXNrs8YZVUxDarVypQiwfi3MWMXgkedEWXcIOTAtBBUNoU8QFuZMIGymN4wyMRpgyhljZUJfqCozBj1KM9DU8hp5Pc9s9EBpJ%2BO3yf8eqO03%2FR9aUnyHGfY2QA39cor952%2FOBB1VUCVbiY4ihdxveuVGSeNiOZzN1OVhip6%2B%2B2fXdJ6aA4XL7y49q6KJMj45EepWHfXifHmBDUI%2F3AD1d%2B3%2BDyFWBzqEqK0YbNXhgL3sm2j8KsmFY%2FjSq7cmRMzmBdTTQn%2FwKz05z4Nc6t6PwM8gamJh5vDCDYRQHTPY2NletI6oQZQYyLeRfpAHSoLBFlqgFIas%2FRHORXFl3ZIP159SPbbsS9kd0NYgDYlgAQOVZat%2FBgCuvGLdYJLO9ZDW00x13pYga7lZ%2BGi0LG0qNm5%2Fzfyhkv42Di%2BGnaDBL%2B0CVCWPtAKKoW2qSHCRuhPWU%2B0R2Pq6dW1AQ00J7SSDpWYi%2Ff%2BWWMT4dE%2FU00jS6TeHc7CvcK5ma%2FBBwJeF0RvKDlUgE8wBCK4NDP0cOsN5qkdlvjnptU4ACz2G2FECnYbbwFs%2Fh7q3z3j%2BT4TFT5OQBHmMuEUq2Ez%2FEc%2FQcHqyPkz0NLXDSoil%2FYHXpcRO%2FpF3AT5dCvK1k8ME%2BByuLtwMv8Ysx6EiCIRUzt%2FCAtFmTYHMn0vtIuZJ3RzItYQ%2Fn9AP3WS%2FGvy48PaHbSMRE1aqCocrF294NtSMPH8pZn2fgZN8TL82iqNiCVmjNusOwDXIyemWJqwjURnrpGST01Uz%2F8EXa51VfJ%2Bq6%2FWB89zgIfdtsKFBdT50PYPlPGW3RnVIXtNqq0y17Yd%2FWcQjU8KwoG79jPHOD1VmNTYbfMAAmj5pmk98rVpkQCjgShkLMdgXqpR6xuVuU05FIZ9iB%2Bwmtjc7PkD9iin9BIz3lys3ieJ8lUbP%2Bww4MjktAY6qAE5thsN02OXfxiuraGEy3kzmgki8HqDlO5maGS3I3pa6ZLQcYRVpdSo4XBbQnyh8J1IwrUmcK3YPwXMZBLfAsMRECB12KKFwjH8hRQdcs2djqBqKT63kTrymZ21SpzOxr8TPw1dSl3GEpVNWQ9xtuttzS2kkmDT6MfqBZGF2oBOMO1gnJ%2BfcalZdgsDqiCVH7cm%2Fx0fvwPe6iuxCHbefyW9HDyc1B84xzA%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240718T151453Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=ASIA4H5P3Z2RCNHWJUDF%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=f9c4198451d382120ba9517c5644a836940ab4de5708b5787c7fe8d2da9f4475)
2024-07-18 15:15:04 source > Backing off _try_read_records(...) for 3.7s (source_amazon_ads.streams.report_streams.report_streams.ReportGenerationFailure: 400 Client Error: Bad Request for url: https://offline-report-storage-us-east-1-prod.s3.amazonaws.com/0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029/report-0d8cf316-d2bb-4db2-bf23-5f4fe10b70d4-1721315019029.json.gz?X-Amz-Security-Token=IQoJb3JpZ2luX2VjED8aCXVzLWVhc3QtMSJIMEYCIQCBaeaTnmm%2F%2FIt3q3Am0mNzyxrRzolA0Y2fhvgM3LZESAIhANKTjrs2Yt9PXJc9%2FuAkfTjSppk58FZjCY3NoV0xdznkKuEFCBcQAhoMODQxNjQ1NTQzMDc0IgziqpiXotr%2FVafuD4gqvgUcDuKSrJlwtdXNrs8YZVUxDarVypQiwfi3MWMXgkedEWXcIOTAtBBUNoU8QFuZMIGymN4wyMRpgyhljZUJfqCozBj1KM9DU8hp5Pc9s9EBpJ%2BO3yf8eqO03%2FR9aUnyHGfY2QA39cor952%2FOBB1VUCVbiY4ihdxveuVGSeNiOZzN1OVhip6%2B%2B2fXdJ6aA4XL7y49q6KJMj45EepWHfXifHmBDUI%2F3AD1d%2B3%2BDyFWBzqEqK0YbNXhgL3sm2j8KsmFY%2FjSq7cmRMzmBdTTQn%2FwKz05z4Nc6t6PwM8gamJh5vDCDYRQHTPY2NletI6oQZQYyLeRfpAHSoLBFlqgFIas%2FRHORXFl3ZIP159SPbbsS9kd0NYgDYlgAQOVZat%2FBgCuvGLdYJLO9ZDW00x13pYga7lZ%2BGi0LG0qNm5%2Fzfyhkv42Di%2BGnaDBL%2B0CVCWPtAKKoW2qSHCRuhPWU%2B0R2Pq6dW1AQ00J7SSDpWYi%2Ff%2BWWMT4dE%2FU00jS6TeHc7CvcK5ma%2FBBwJeF0RvKDlUgE8wBCK4NDP0cOsN5qkdlvjnptU4ACz2G2FECnYbbwFs%2Fh7q3z3j%2BT4TFT5OQBHmMuEUq2Ez%2FEc%2FQcHqyPkz0NLXDSoil%2FYHXpcRO%2FpF3AT5dCvK1k8ME%2BByuLtwMv8Ysx6EiCIRUzt%2FCAtFmTYHMn0vtIuZJ3RzItYQ%2Fn9AP3WS%2FGvy48PaHbSMRE1aqCocrF294NtSMPH8pZn2fgZN8TL82iqNiCVmjNusOwDXIyemWJqwjURnrpGST01Uz%2F8EXa51VfJ%2Bq6%2FWB89zgIfdtsKFBdT50PYPlPGW3RnVIXtNqq0y17Yd%2FWcQjU8KwoG79jPHOD1VmNTYbfMAAmj5pmk98rVpkQCjgShkLMdgXqpR6xuVuU05FIZ9iB%2Bwmtjc7PkD9iin9BIz3lys3ieJ8lUbP%2Bww4MjktAY6qAE5thsN02OXfxiuraGEy3kzmgki8HqDlO5maGS3I3pa6ZLQcYRVpdSo4XBbQnyh8J1IwrUmcK3YPwXMZBLfAsMRECB12KKFwjH8hRQdcs2djqBqKT63kTrymZ21SpzOxr8TPw1dSl3GEpVNWQ9xtuttzS2kkmDT6MfqBZGF2oBOMO1gnJ%2BfcalZdgsDqiCVH7cm%2Fx0fvwPe6iuxCHbefyW9HDyc1B84xzA%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240718T151453Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=ASIA4H5P3Z2RCNHWJUDF%2F20240718%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=f9c4198451d382120ba9517c5644a836940ab4de5708b5787c7fe8d2da9f4475)
2024-07-18 15:15:07 source > Checking report status, 6 report(s) remaining

Contribute

• Yes, I want to contribute

[SalvitSimmus7]

update: I read the code of amazon-ads sources and the problem doesn't seem to be the timeout. In the logs it seems that it is waiting enough until the reports are generated. I think the problem is related to the process of downloading the reports once they are completed

def _download_report(self, report_info: ReportInfo, url: str) -> List[dict]:
        """
        Download and parse report result
        """
        response = self._send_http_request(url, report_info.profile_id) if report_info else self._send_http_request(url, None)
        response.raise_for_status()
        raw_string = decompress(response.content).decode("utf")
        return json.loads(raw_string)

This function inside the amazon-ads source triggers this other function

def _send_http_request(self, url: str, profile_id: int, json: dict = None):
        headers = self._get_auth_headers(profile_id)
        if json:
            response = self._session.post(url, headers=headers, json=json)
        else:
            response = self._session.get(url, headers=headers)
        if response.status_code == HTTPStatus.TOO_MANY_REQUESTS:
            raise TooManyRequests()
        return response

def _get_auth_headers(self, profile_id: int):
        return (
            {
                "Amazon-Advertising-API-ClientId": self._client_id,
                "Amazon-Advertising-API-Scope": str(profile_id),
                **self._session.auth.get_auth_header(),
            }
            if profile_id
            else {}
        )

What could be wrong with my source config that is making one of this 3 functions to have a non-expected behavior? Why is it that the other reports work (like sponsored_brands_report_stream and sponsored_display_report_stream) and they are indeed downloading the reports successfully?

[benjamin-steurer]

Having the same issue here