Skip to content

genai: Enhance retry mechanism in ChatGoogleGenerativeAI with customizable parameter #915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

genai: Enhance retry mechanism in ChatGoogleGenerativeAI with customizable parameter #915

wants to merge 2 commits into from

Conversation

bajajku
Copy link

@bajajku bajajku commented May 9, 2025

PR Description

  1. Fixed the retry mechanism to properly use the max_retries parameter instead of hardcoding to 2
  2. Made backoff parameters configurable:
    • wait_exponential_multiplier (default: 2.0)
    • wait_exponential_min (default: 1.0)
    • wait_exponential_max (default: 60.0)
  3. Added handling of quota-exceeded errors to use recommended retry delay when provided

Relevant issues

Fixes #857

Type

🆕 New Feature
🐛 Bug Fix

Testing

Added unit tests to verify: Configurable retry parameters are respected

@bajajku
Enhance retry mechanism in ChatGoogleGenerativeAI with customizable p…
4141164 
…arameters

- Updated `_create_retry_decorator` to accept parameters for max retries and exponential backoff settings.
- Modified `_chat_with_retry` to utilize these parameters from `kwargs`.
- Added handling for `ResourceExhausted` exceptions to respect the `retry_after` delay.
- Introduced a new test to validate the retry decorator with custom parameters.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lkuligin lkuligin lkuligin approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

max_retries and backoff config not applied; improve retry logic for quota errors
2 participants
@bajajku @lkuligin