-
Notifications
You must be signed in to change notification settings - Fork 990
Added support for google specific arguments for video analysis #2110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
I'm still not exactly sure why are those tests failing, so I would be thankful if you could provide me with an explanation on what to do with those? |
start_offset=item.vendor_metadata.get('start_offset', None), | ||
end_offset=item.vendor_metadata.get('end_offset', None), | ||
) | ||
inline_data_dict['video_metadata'] = video_metadata # type: ignore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would this work?
inline_data_dict['video_metadata'] = video_metadata # type: ignore | |
inline_data_dict['video_metadata'] = item.vendor_metadata # type: ignore |
vendor_metadata: dict[str, Any] | None = None | ||
"""The vendor specific metadata for the file. | ||
Currently supports only those keys: | ||
|
||
fps: float, | ||
start_offset: str (ex. 1800s), | ||
end_offset: str (ex. 1800s) | ||
|
||
And works only for google models for video analysis. | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer something like this:
vendor_metadata: dict[str, Any] | None = None | |
"""The vendor specific metadata for the file. | |
Currently supports only those keys: | |
fps: float, | |
start_offset: str (ex. 1800s), | |
end_offset: str (ex. 1800s) | |
And works only for google models for video analysis. | |
""" | |
vendor_metadata: dict[str, Any] | None = None | |
"""Vendor-specific metadata for the file. | |
Supported by: | |
- `GoogleModel`: `VideoUrl.vendor_metadata` is used as `video_metadata`: https://ai.google.dev/gemini-api/docs/video-understanding#customize-video-processing | |
""" |
@@ -1191,20 +1191,21 @@ wheels = [ | |||
|
|||
[[package]] | |||
name = "google-genai" | |||
version = "1.15.0" | |||
version = "1.23.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test failure suggests that with the genai update, the previously recorded cassettes don't match anymore because post != POST
... Can you try replacing method: POST
with method: post
in all the tests/models/cassettes/test_google/*.yaml
files?
Added support for Google models specific arguments when processing video, those arguments are:
Official Google docs for those new arguments:
https://ai.google.dev/gemini-api/docs/video-understanding