Ideas Log for Home Assistant Custom Integration

[ITSpecialist111]

Adding the suggestions here from the BreadedTinker YouTube video comments:

Automation Suggestions for Existing Devices
Contributor: @Dominoxan
Summary: The feature to "suggest automations based on newly added devices" raised the question of how existing devices will be handled. @Dominoxan asks if users need to re-add their integrations to benefit from automation suggestions for existing devices.
Status: Pending Investigation / Enhancement Discussion

Data Handling and Local Large Language Model (LLM) Support
Contributor: @Dominoxan
Summary: If the integration evolves to handle large data volumes, particularly from SQL logs, local LLM options like Ollama or LMStudio could be useful to manage data without sending it to the cloud.
Status: Feasibility Review

Billing Model for API Queries
Contributor: @geejayem59
Summary: Clarification is requested on how billing works for API usage—whether charges occur per query or if there's an upfront payment with incremental deductions. This could be relevant for managing costs effectively.
Status: Documentation Update Needed

Usage Patterns for Automation Suggestions
Contributor: @brianmorgan7207
Summary: At a recent HA gathering, @brianmorgan7207 discussed the concept of AI suggesting automations based on usage patterns (who interacts with which entities). They view this feature as a valuable addition to enhance the AI Assistant’s functionality.
Status: Feature Request

Guidance on API Key Options for Beginners
Contributor: @scienceguyroy (with additional input from @grahamahosking)
Summary: A recommendation or guide on selecting an API key for new users would be beneficial. Additionally, @grahamahosking suggests local LLMs as an alternative for users concerned with costs, noting that response speed may be less critical for this integration.
Status: Documentation Update / Resource Link Addition

[ITSpecialist111]

High-Priority Enhancements:

1. Model Selection Flexibility:

   • Implement a drop-down menu allowing users to choose between different generative AI models, including Google Generative AI and OpenAI's GPT models. This will address user requests for more options and increase flexibility.

2. Token Limit Customization:

   • Extend the maximum token limit to 16,384 tokens and make it user-configurable. This will help support larger Home Assistant setups and prevent data cut-off issues.

3. Splitting Notifications:

   • Modify the current notification system to split suggestions into separate items. This will let users dismiss irrelevant suggestions and keep only the ones they find useful.

4. Enhanced Suggestion Formatting:

   • Ensure that all AI automation suggestions are presented in a structured format with a title, trigger, action, and YAML code sample for each suggestion. This format will make it easier for users to understand and implement the automations.

Medium-Priority Features:

5. Local LLM Support and Data Handling:

   • Explore the feasibility of supporting local large language models (e.g., Ollama or LMStudio) to handle data without using cloud services. This would appeal to users concerned about data privacy and cost management.

6. Swipe-to-Acknowledge Feature:

   • Implement the planned feature that allows users to swipe to acknowledge or dismiss suggestions. This will improve user interaction with notifications.

Documentation and Guidance:

7. API Key Guidance:

   • Update the documentation to include a guide for selecting an API key and understanding billing models. This will help users manage costs and make informed decisions.

8. Automation Suggestions for Existing Devices:

   • Provide clarity or implement a solution that allows automation suggestions for existing devices without re-adding integrations. This will enhance the overall functionality of the integration.

Future Considerations:

9. Cost Management and Billing Model Clarification:

   • Clarify the billing model for API usage in the documentation, specifying whether charges occur per query or are part of a prepaid plan.

10. Usage Pattern Analysis:

    • Investigate adding functionality to suggest automations based on usage patterns (e.g., who interacts with specific entities). This would make the AI assistant more context-aware and valuable.

[ITSpecialist111]

Completed 1st high priority task above with additions:

V1.07 Now Provides these asks via manual service calls.

Removal of Automatic Scheduling:

Removed automatic scanning and suggestion generation due to complexities with triggering and user preferences.
Enhanced Stability: This change improves code stability and allows users to implement custom automation triggers that suit their individual needs.

Expanded AI Provider Support:

Added Support for Multiple LLM Models:
Google LLM Models: Now supports Google's AI models for generating automation suggestions.
Local AI Models: Introduced support for local models like LocalAI and Ollama for users who prefer privacy and do not rely on internet connectivity.

Custom LLM Variants:
Users can now choose custom AI model variants from higher models (e.g., OpenAI's o1-preview), providing more flexibility and control over the AI processing.
Enhanced Manual Trigger Service:

Integration Selection: Users can now select which AI provider integration to trigger when manually initiating the AI analysis.
Custom System Prompt: Allows users to override the built-in system prompt for more granular and personalized suggestions.
German Translation Added:

Global Reach: Introduced German language support to make the integration accessible to a wider audience.