Skip to content

Gemini does not detect video or voice. #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Toltally-suck-at-code opened this issue May 28, 2025 · 3 comments
Open

Gemini does not detect video or voice. #108

Toltally-suck-at-code opened this issue May 28, 2025 · 3 comments
Labels
bug Something isn't working

Comments

@Toltally-suck-at-code
Copy link

Description of the bug:

Hi there!
When running this app, whenever I turn on my mic and my webcam, it doesn't respond anything to me, I checked the logs, and see that it's repeatedly sending video data and preventing from the actual API to process

Here's the log in the app provides

8:26:51client.openConnected
8:26:51server.sendsetupComplete
8:26:55client.realtimeInputaudio39
8:26:55client.realtimeInputvideo
8:26:57client.realtimeInputaudio31
8:26:57client.realtimeInputvideo
8:26:59client.realtimeInputaudio29
8:26:59client.realtimeInputvideo
8:27:07client.realtimeInputaudio248
8:27:07client.realtimeInputvideo
8:27:08client.realtimeInputaudio45

When turning off the webcam and the mic, and only using pure text, it responds normally, but seems to hallucinate and tells me that it hears me and sees my video feed???

Secondly, when using the direct SDK provided by Google Studio AI, sometimes it seems to also hallucinate and pretends that it doesn't hear me and doesn't see my webcam (even though I am speaking to it and streaming via OpenCV) (PS The second issue, I used the latest 2.5 native thinking audio model, when I switch back to the older 2.0 Version, it stops hallucinating. And yes, I tried changing API keys and restarting to try to "reset" the chatlogs (Or tried to since it still remember what I have said earlier???)

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

@Toltally-suck-at-code
Copy link
Author

Update, when turning only the webcam on and the mic on while texting the AI instead of speaking, it responds normally....

Can I implement some type of button that manually stops the video feed and mic feed so it wont keep sending Video and Mic requests to the Gemini Live Web Kit???

@hapticdata
Copy link
Member

Hi! when did you clone the repo? A fix for this was pushed yesterday e4de0d3

@hapticdata hapticdata added the bug Something isn't working label May 28, 2025
@Toltally-suck-at-code
Copy link
Author

Toltally-suck-at-code commented May 29, 2025

Hi Kyle, I cloned the repo around 30 minutes before I posted the bug issue

Edit: "bug issue" not "bug fix" sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants