ChatVertexAI InvalidArgument Error (Missing Parts Field) with Large Context in System Message (Gemini 2.5 Flash Preview Model)

[macgirlsweden]

We are sending JSON-data from our text extraction process in a SystemMessage and getting this error:

Here is the error message:

WARNING:langchain_google_vertexai._retry:Retrying langchain_google_vertexai.chat_models._completion_with_retry.._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
WARNING:langchain_google_vertexai._retry:Retrying langchain_google_vertexai.chat_models._completion_with_retry.._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
WARNING:langchain_google_vertexai._retry:Retrying langchain_google_vertexai.chat_models._completion_with_retry.._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
WARNING:langchain_google_vertexai._retry:Retrying langchain_google_vertexai.chat_models._completion_with_retry.._completion_with_retry_inner in 8.0 seconds as it raised InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
WARNING:langchain_google_vertexai._retry:Retrying langchain_google_vertexai.chat_models._completion_with_retry.._completion_with_retry_inner in 10.0 seconds as it raised InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini.
ERROR:main:Error processing request: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini

Environment:
Framework: LangChain (using LangGraph StateGraph with MessagesState)
Model Integration: langchain_google_vertexai.ChatVertexAI
Model Name: Primarily failing with gemini-2.5-flash-preview-04-17.
Platform: Flask application running on Google Cloud, using Firestore for chat history and GCS for document context storage.

LangChain Libraries:
langchain==0.3.24
langchain-community==0.3.1
langchain-core==0.3.56
langchain-google-vertexai==2.0.21
langchain-openai==0.2.2
langchain-pinecone==0.2.0
langchain-text-splitters==0.3.8
langgraph==0.2.34
langgraph-checkpoint==2.0.0
langsmith==0.1.131
Python Version: 3.9.22

Application Goal & Workflow:

• I have built a single-document Q&A chatbot endpoint. The core workflow is as follows:
• Load previous chat history (Q&A pairs stored in Firestore) into a list of HumanMessage and AIMessage objects using a custom load_chat_history function.
• Load the relevant document context from Google Cloud Storage. This context is pre-processed text formatted as a JSON string (potentially large, containing a list of text chunks with metadata) and stored in a Python variable named text.
• Use a ChatPromptTemplate pulled from LangSmith Hub (chatdoc-parsd-dev). This template is structured with:
• A SystemMessage containing persona instructions, general rules, and a placeholder for the document context ({text}), which is delimited using json ... .
• A MessagesPlaceholder("messages") intended for the chat history.
• Within a LangGraph call_model node (defined inside the Flask route):
• The current user question is appended as a HumanMessage to the loaded history list (state["messages"]).
• The combined message list is trimmed using langchain_core.messages.trim_messages (currently configured with max_tokens=5, strategy="last", include_system=True - planning to adjust to max_messages=5 and include_system=False based on debugging).
• The final list of messages for the model is prepared using prompt_template.format_messages(messages=trimmed_history, text=text, role=..., purpose=..., ...).
• The model is invoked using model.invoke(prompt_messages). This is where the error occurs.
• The response (if successful) is processed and saved back to Firestore.
• The Problem & Key Debugging Findings:
• The application works correctly for many documents - The failure occurs only with certain documents. Debug logs show the problematic context string has ~120k tokens (calculated using tokenizer.count_tokens), with the total input tokens being slightly higher (~120k). This is well within the 1M token limit for the gemini-1.5-flash-001 tokenizer used for the check (and the target gemini-2.5-flash-preview-04-17 model).
• The specific error received during the model.invoke() call is consistently:
  - google.api_core.exceptions.InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
• LangChain's built-in retry mechanism (visible in logs) also fails with the same InvalidArgument error.

      
I have updated to the recent chatvertextai, langchain-core and langchain version.

Debugging steps taken:

• JSON Validation: Added json.loads(text) validation immediately after loading the context from GCS. It succeeds for the failing document, confirming the large JSON string is structurally valid. Log message: DEBUG: JSON string is structurally valid.
• History Loading: Confirmed load_chat_history returns a flat list [HumanMessage, AIMessage, ...].
• Input Structure to model.invoke: Added detailed logging right before the model.invoke(prompt_messages) call. This confirmed:
• type(prompt_messages) is <class 'list'>.
• The structure is a flat list like [SystemMessage(...), HumanMessage(...), AIMessage(content=''), HumanMessage(...)]. The previously suspected [[...]] nesting issue is not present at this point.
• Empty AIMessage content in the history is present but considered unlikely to cause this specific API error.
• JSON files that is used to create the JSON string has content-type: application/json; charset=utf-8

Current Hypothesis:

• Given that the input prompt_messages list passed to model.invoke has the correct flat structure, the embedded JSON context (text) within the SystemMessage is structurally valid JSON, and the total token count is within limits, the error strongly suggests a failure during the serialization step within ChatVertexAI.invoke() or the underlying Google client libraries.
• It appears the process of converting the LangChain message list into the specific Vertex AI API JSON payload (which requires a parts field for content) fails when the SystemMessage contains extremely large content (like the ~120k token JSON string), even if the overall token count is acceptable.

[macgirlsweden]

Here is the full error message:

`InvalidArgument('Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini')Traceback (most recent call last):

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/grpc/_channel.py", line 1181, in call
return _end_unary_response_blocking(state, call, False, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini"
debug_error_string = "UNKNOWN:Error received from peer ipv4:216.58.206.74:443 {created_time:"2025-05-05T22:30:27.518452242+00:00", grpc_status:3, grpc_message:"Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini"}"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langgraph/pregel/init.py", line 1298, in stream
for _ in runner.tick(

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langgraph/pregel/runner.py", line 56, in tick
run_with_retry(t, retry_policy)

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langgraph/pregel/retry.py", line 29, in run_with_retry
task.proc.invoke(task.input, config)

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langgraph/utils/runnable.py", line 409, in invoke
input = context.run(step.invoke, input, config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langgraph/utils/runnable.py", line 183, in invoke
ret = context.run(self.func, input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/workspace/main.py", line 1679, in call_model
response = model.invoke(prompt_messages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 284, in invoke
self.generate_prompt(

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 784, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 641, in generate
raise e

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 631, in generate
self._generate_with_cache(

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 850, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_google_vertexai/chat_models.py", line 1174, in _generate
return self._generate_gemini(
^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_google_vertexai/chat_models.py", line 1331, in _generate_gemini
response = _completion_with_retry(
^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_google_vertexai/chat_models.py", line 612, in _completion_with_retry
return _completion_with_retry_inner(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/tenacity/init.py", line 336, in wrapped_f
return copy(f, *args, **kw)
^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/tenacity/init.py", line 475, in call
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/tenacity/init.py", line 376, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/tenacity/init.py", line 418, in exc_check
raise retry_exc.reraise()
^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/tenacity/init.py", line 185, in reraise
raise self.last_attempt.result()
^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.runtime/python/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.runtime/python/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/tenacity/init.py", line 478, in call
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/langchain_google_vertexai/chat_models.py", line 605, in _completion_with_retry_inner
return generation_method(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/google/cloud/aiplatform_v1beta1/services/prediction_service/client.py", line 2280, in generate_content
response = rpc(
^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in call
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/layers/google.python.pip/pip/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc

google.api_core.exceptions.InvalidArgument: 400 Unable to submit request because it must include at least one parts field, which describes the prompt input. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini`