Templated agents utilize OpenTelemetry and OpenLLMetry for comprehensive observability, emitting events to Google Cloud Trace and Google Cloud Logging. Every interaction with LangChain and VertexAI is instrumented (see server.py), enabling detailed tracing of request flows throughout agents built with this framework.
Leveraging the CloudTraceSpanExporter, the framework captures and exports tracing data. To address the limitations of Cloud Trace (256-byte attribute value limit) and Cloud Logging (256KB log entry size), a custom extension of the CloudTraceSpanExporter is implemented in app/utils/tracing.py.
This extension enhances observability by:
- Creating a corresponding Google Cloud Logging entry for every captured event.
- Automatically storing event data in Google Cloud Storage when the payload exceeds 256KB.
Logged payloads are associated with the original trace, ensuring seamless access from the Cloud Trace console.
Events are forwarded to BigQuery through a log router for long-term storage and analysis. The deployment of the log router is handled via Terraform code in deployment/terraform.
Once the data is written to BigQuery, it can be used to populate a Looker Studio dashboard.
This dashboard template provides a starting point for building custom visualizations on top of the captured data.
Note: The templated agents are designed to enable your use-case observability in your Google Cloud Project. Google Cloud does not log, monitor, or otherwise access any data generated from the deployed resources. See the Google Cloud Service Terms for more details.