Made Google Search enable dependent on Assist availability

[IvanLH]

Proposed change

Changes the Options form to make Google Search enablement show an error if Assist is also set

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Deprecation (breaking change to happen in the future)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request: Added documentation for Google Search option home-assistant.io#38363
• Link to developer documentation pull request:
• Link to frontend pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• I have followed the perfect PR recommendations
• The code has been formatted using Ruff (ruff format homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[home-assistant]

Hey there @tronikos, mind taking a look at this pull request as it has been labeled with an integration (google_generative_ai_conversation) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of google_generative_ai_conversation can trigger bot actions by commenting:

• @home-assistant close Closes the pull request.
• @home-assistant rename Awesome new title Renames the pull request.
• @home-assistant reopen Reopen the pull request.
• @home-assistant unassign google_generative_ai_conversation Removes the current integration label and assignees on the pull request, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

[Andrerm124]

Apologies if this is not the right place, but is there a specific reason that we can't use Google Search alongside the Assist API?
The ability to have an AI assistant, that can have up to date factual information, along with controlling your smart home feels like the goal, doesn't it?

[balloob]

It's a limitation from the Google API.

[tronikos]

Can you add a data_description for enable_google_search_tool mentioning the "No control" requirement?

[tronikos]

Apologies if this is not the right place, but is there a specific reason that we can't use Google Search alongside the Assist API? The ability to have an AI assistant, that can have up to date factual information, along with controlling your smart home feels like the goal, doesn't it?

A workaround is to create a separate agent with search tool enabled and create the following automation:

alias: "Assist: Ask Google"
description: ""
triggers:
  - command: ask google {req}
    trigger: conversation
conditions: []
actions:
  - action: conversation.process
    metadata: {}
    data:
      text: "{{ trigger.slots.req }}"
      agent_id: conversation.google_generative_ai_search
    response_variable: resp
  - set_conversation_response: "{{ resp.response.speech.plain.speech }}"
mode: single

Then in your regular agent with assist enabled (either Google Generative AI or local agent) say: "ask Google what's today's tech news?"

[Andrerm124]

@tronikos Amazing, that's a great idea!

[Andrerm124]

Just altered one of my other scripts to do this.
Might be a neater solution as it allows the LLM to act on the information returned 🙂

sequence:
  - action: conversation.process
    metadata: {}
    data:
      agent_id: conversation.google_generative_ai_2
      text: "{{ query }}"
    response_variable: result
  - variables:
      result:
        response: "{{ result.response.speech.plain.speech }}"
  - stop: ""
    response_variable: result
alias: "Assist: Search Google"
description: >-
  Search Google for specific real time information. Use this script when you
  need to gather information about a topic that may require up to date
  information. 
fields:
  query:
    selector:
      text: null
    name: Query
    description: The query to search Google for
    required: true

Note: I'll likely tweak the script Description to be better suited

[tronikos]

@Andrerm124 Nice! Can you add it to the docs?

[swever826]

Just altered one of my other scripts to do this.
Might be a neater solution as it allows the LLM to act on the information returned 🙂

sequence:
  - action: conversation.process
    metadata: {}
    data:
      agent_id: conversation.google_generative_ai_2
      text: "{{ query }}"
    response_variable: result
  - variables:
      result:
        response: "{{ result.response.speech.plain.speech }}"
  - stop: ""
    response_variable: result
alias: "Assist: Search Google"
description: >-
  Search Google for specific real time information. Use this script when you
  need to gather information about a topic that may require up to date
  information. 
fields:
  query:
    selector:
      text: null
    name: Query
    description: The query to search Google for
    required: true

Note: I'll likely tweak the script Description to be better suited

Thanks for sharing your script!! I also had a script before this was implemented and I used the Google Custom Search API. Glad that's not needed now, since this implementation gives better results.

I just wanted to share that your script description wasn't enough for the LLM to use the script (at least in my HA) so let me share the description I used in my old script and it seems to work great! I wanted for the LLM to search the web only when truly needed and this is what has worked better for me. Let me know if it helps!

description: >-
  Makes a Google search to answer questions that are completely unrelated with
  the smart home and are exclusively about current events or information in
  real-time like the current president, results of last night's game, release
  dates, etc.

[ndom91]

Thanks for sharing your workarounds everyone! I've now had a good experience leaving my default OpenAI conversation with Control, and added antoher Google -> Generative AI conversation integration with its control disabled, but web search enabled and the following script (dont forget to expose the script to voice assistants!)

sequence:
  - action: conversation.process
    metadata: {}
    data:
      agent_id: conversation.google_generative_ai
      text: "{{ query }}"
    response_variable: result
  - variables:
      result:
        response: "{{ result.response.speech.plain.speech }}"
  - stop: ""
    response_variable: result
alias: "Assist: Search Google"
description: >-
  Makes a Google search to answer questions that are completely unrelated with
  the smart home and are exclusively about current events or information in
  real-time like the current president, release dates, etc.
fields:
  query:
    selector:
      text: null
    name: Query
    description: The query to search Google for
    required: true