docs(generative_ai): Update RAG Engine Samples to V1

[holtskinner]

Description

Code Samples were moved to Hard-Coded in Docs by Product team when GA launched

https://cloud.google.com/vertex-ai/generative-ai/docs/rag-overview

[gemini-code-assist]

Hello @holtskinner, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request updates the RAG (Retrieval-Augmented Generation) engine samples to use the V1 version of the Vertex AI API. This involves changes to import statements, configuration of embedding models, and the structure of parameters passed to the create_corpus, import_files, and retrieval_query functions. The pull request also updates the Gemini model name from gemini-1.5-flash-001 to gemini-2.0-flash-001.

Highlights

• API Version Update: The samples are updated to use the V1 version of the Vertex AI API, replacing vertexai.preview with vertexai.
• Embedding Model Configuration: The configuration of embedding models has been updated to use RagEmbeddingModelConfig and VertexPredictionEndpoint.
• Function Parameter Changes: The parameters passed to create_corpus, import_files, and retrieval_query functions have been updated to align with the V1 API.
• Model Name Update: The Gemini model name has been updated from gemini-1.5-flash-001 to gemini-2.0-flash-001.
• Chunking Configuration: The import_files function now uses transformation_config with rag.ChunkingConfig to specify chunk size and overlap.

Changelog

Click here to see the changelog

• generative_ai/rag/create_corpus_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Replaced embedding_model_config with backend_config using RagVectorDbConfig and RagEmbeddingModelConfig.
  • Updated the publisher_model to publishers/google/models/text-embedding-005.
• generative_ai/rag/create_corpus_pinecone_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated embedding_model_config to use RagEmbeddingModelConfig and VertexPredictionEndpoint.
  • Wrapped embedding_model_config and vector_db in backend_config using RagVectorDbConfig.
  • Updated the publisher_model to publishers/google/models/text-embedding-005.
• generative_ai/rag/create_corpus_vector_search_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated embedding_model_config to use RagEmbeddingModelConfig and VertexPredictionEndpoint.
  • Wrapped embedding_model_config and vector_db in backend_config using RagVectorDbConfig.
  • Updated the publisher_model to publishers/google/models/text-embedding-005.
• generative_ai/rag/create_corpus_vertex_ai_search_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated return type annotation to rag.RagCorpus.
• generative_ai/rag/delete_corpus_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
• generative_ai/rag/delete_file_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
• generative_ai/rag/generate_content_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated import statement from vertexai.preview.generative_models to vertexai.generative_models.
  • Replaced similarity_top_k and vector_distance_threshold with rag_retrieval_config using RagRetrievalConfig and rag.utils.resources.Filter.
  • Updated the model_name to gemini-2.0-flash-001.
• generative_ai/rag/get_corpus_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
• generative_ai/rag/get_file_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated import statement from google.cloud.aiplatform_v1beta1 to google.cloud.aiplatform_v1.
• generative_ai/rag/import_files_async_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated import statement from google.cloud.aiplatform_v1beta1 to google.cloud.aiplatform_v1.
  • Replaced chunk_size and chunk_overlap with transformation_config using rag.TransformationConfig and rag.ChunkingConfig.
  • Replaced rag.import_files_async with rag.import_files.
• generative_ai/rag/import_files_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated import statement from google.cloud.aiplatform_v1beta1 to google.cloud.aiplatform_v1.
  • Replaced chunk_size and chunk_overlap with transformation_config using rag.TransformationConfig and rag.ChunkingConfig.
• generative_ai/rag/list_corpora_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
• generative_ai/rag/list_files_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated import statement from google.cloud.aiplatform_v1beta1 to google.cloud.aiplatform_v1.
• generative_ai/rag/quickstart_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Removed import of RagCorpus from vertexai.preview.rag.utils.resources.
  • Updated embedding_model_config to use RagEmbeddingModelConfig and VertexPredictionEndpoint.
  • Wrapped embedding_model_config in backend_config using RagVectorDbConfig.
  • Replaced chunk_size and chunk_overlap with transformation_config using rag.TransformationConfig and rag.ChunkingConfig.
  • Replaced similarity_top_k and vector_distance_threshold with rag_retrieval_config using RagRetrievalConfig and rag.Filter.
  • Updated the model_name to gemini-2.0-flash-001.
  • Updated the publisher_model to publishers/google/models/text-embedding-005.
• generative_ai/rag/rag_test.py
  • Removed skip decorator.
  • Added skip decorators to tests that require external resources.
• generative_ai/rag/requirements.txt
  • Updated google-cloud-aiplatform version from 1.74.0 to 1.87.0 and removed the [all] extra.
• generative_ai/rag/retrieval_query_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Replaced similarity_top_k and vector_distance_threshold with rag_retrieval_config using RagRetrievalConfig and rag.utils.resources.Filter.
• generative_ai/rag/upload_file_example.py
  • Updated import statement from vertexai.preview import rag to vertexai import rag.
  • Updated return type annotation to rag.RagFile.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

The RAG evolves, a new version,
From preview's realm to V1's conversion.
Models updated, configs refined,
A better engine, for queries designed.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request updates the RAG Engine samples to V1, which involves changes to import paths, configuration structures, and model names. The code has been updated to reflect the new API, and the changes seem reasonable. However, there are a few areas that could be improved for clarity and consistency.

Summary of Findings

• API Versioning: The update involves migrating from vertexai.preview.rag to vertexai.rag. This change should be reflected consistently across all files to ensure the examples are using the correct API version.
• Configuration Structure: The configuration of embedding models and vector databases has changed. Ensure that the new configuration structure is correctly implemented and that all necessary parameters are set.
• Model Names: The default model name has been updated from gemini-1.5-flash-001 to gemini-2.0-flash-001. Verify that this change is intentional and that the new model is appropriate for the examples.
• Test Skipping: Several tests are marked as skipped. It's important to ensure that these tests are either updated to work with the new API or that there is a valid reason for skipping them.

Merge Readiness

The pull request updates the RAG Engine samples to V1. While the changes appear to be correct, it's important to ensure that all tests are passing and that the new API is being used consistently across all examples. I recommend addressing the comments before merging. I am unable to directly approve this pull request, and other reviewers should review and approve this code before merging.