Get Snowflake + Hamilton UI example ready

Note: this is a toy example. For real production needs, you'd need
to modify a few things:

1. not use SQLLITE, and instead postgresql, or implement django-snowflake connection.
2. likely not use the Hamilton code within the flask container, instead package up properly
and define a UDF or UDTF.
3. could use snowpark dataframes or have hamilton code do other things, e.g. LLM calls..

Author: skrawcz

docs/hamilton-ui/ui.rst

@@ -123,6 +123,12 @@ More extensive self-hosting documentation is in the works, e.g. Snowflake, Datab
 chart contribution!
 
 
+Running on Snowflake
+---------------------
+You can run the Hamilton UI on Snowflake Container Services. For a detailed guide, see the blog post
+`Observability of Python code and application logic with Hamilton UI on Snowflake Container Services <https://medium.com/@pkantyka/observability-of-python-code-and-application-logic-with-hamilton-ui-on-snowflake-container-services-a26693b46635>`_ by
+`Greg Kantyka <https://medium.com/@pkantyka>`_ and the `Hamilton Snowflake Example <https://github.com/DAGWorks-Inc/hamilton/tree/main/examples/snowflake/hamilton_ui>`_.
+
 -----------
 Get started
 -----------

examples/snowflake/hamilton_ui/README.md

@@ -1,9 +1,16 @@
 # Running the Hamilton & the Hamilton UI in Snowflake
 
-This example is code for the ["TODO" post]().
+This example is code for the ["Observability of Python code and application logic with Hamilton UI on Snowflake Container Services" post](https://medium.com/@pkantyka/observability-of-python-code-and-application-logic-with-hamilton-ui-on-snowflake-container-services-a26693b46635) by
+[Greg Kantyka](https://medium.com/@pkantyka).
 
-Here we show the code required to be packaged up for
-use on Snowflake.
+Here we show the code required to be packaged up for use on Snowflake:
 
-TODO:
- - cut & paste instructions from blog post?
+1. Docker file that runs the Hamilton UI and a flask endpoint to exercise Hamilton code
+2. my_functions.py - the Hamilton code that is exercised by the flask endpoint
+3. pipeline_endpoint.py - the flask endpoint that exercises the Hamilton code
+
+To run see:
+ - snowflake.sql that contains all the SQL to create the necessary objects in Snowflake and exercise things.
+
+For more details see ["Observability of Python code and application logic with Hamilton UI on Snowflake Container Services" post](https://medium.com/@pkantyka/observability-of-python-code-and-application-logic-with-hamilton-ui-on-snowflake-container-services-a26693b46635) by
+[Greg Kantyka](https://medium.com/@pkantyka).

examples/snowflake/hamilton_ui/pipeline_endpoint.py

@@ -1,3 +1,15 @@
+"""
+This module:
+- Defines a flask app that listens for POST requests on /echo
+- the /echo command is invoked from a Snowflake SQL query
+- the /echo command calls a function get_response that is defined in this module
+- get_response uses Hamilton to execute a pipeline defined in my_functions.py
+- my_functions.py contains the functions that are used in the pipeline
+- the pipeline is executed with the input data from the Snowflake query
+- the output of the pipeline is returned to Snowflake
+- the Hamilton UI tracker is used to track the execution of the pipeline
+"""
+
 import logging
 import os
 import sys
@@ -13,7 +25,7 @@
 from hamilton import driver
 from hamilton_sdk import adapters
 
-# WRAPER CODE FOR SNOWFLAKE FUNCTION ######
+# WRAPPER CODE FOR SNOWFLAKE FUNCTION ######
 
 SERVICE_HOST = os.getenv("SERVER_HOST", "0.0.0.0")
 SERVICE_PORT = os.getenv("SERVER_PORT", 8080)
@@ -42,6 +54,7 @@ def readiness_probe():
 
 @app.post("/echo")
 def echo():
+    """This is the endpoint that Snowflake will call to run Hamilton code."""
     message = request.json
     logger.debug(f"Received request: {message}")
 
@@ -61,17 +74,18 @@ def echo():
     return response
 
 
-# END OF WRAPER CODE FOR SNOWFLAKE FUNCTION ######
+# END OF WRAPPER CODE FOR SNOWFLAKE FUNCTION ######
 
 
 def get_response(prj_id, spend, signups, output_columns):
+    """The function that is called from SQL on Snowflake."""
     tracker = adapters.HamiltonTracker(
         project_id=prj_id,
         username="admin",
         dag_name="MYDAG",
         tags={"environment": "R&D", "team": "MY_TEAM", "version": "Beta"},
     )
-    initial_columns = {  # load from actuals or wherever -- this is our initial data we use as input.
+    input_columns = {
         "signups": pd.Series(spend),
         "spend": pd.Series(signups),
     }
@@ -81,11 +95,11 @@ def get_response(prj_id, spend, signups, output_columns):
         .with_modules(
             my_functions
         )  # we need to tell hamilton where to load function definitions from
-        .with_adapters(tracker)  # we want a pandas dataframe as output
+        .with_adapters(tracker)  # we add the Hamilton UI tracker
         .build()
     )
 
-    df = dr.execute(output_columns, inputs=initial_columns)
+    df = dr.execute(output_columns, inputs=input_columns)
 
     serializable_df = {}
 

examples/snowflake/hamilton_ui/pipeline_endpoint_.py

@@ -0,0 +1,2 @@
+flask
+sf-hamilton[ui,sdk]

examples/snowflake/hamilton_ui/requirements.txt

@@ -1,4 +1,5 @@
-https://medium.com/@pkantyka/observability-of-python-code-and-application-logic-with-hamilton-ui-on-snowflake-container-services-a26693b46635
+-- For more details visit:
+-- https://medium.com/@pkantyka/observability-of-python-code-and-application-logic-with-hamilton-ui-on-snowflake-container-services-a26693b46635
 
 CREATE SERVICE public.hamilton_ui
 IN COMPUTE POOL TEST_POOL
@@ -90,4 +91,4 @@ FROM
   flattened f
   left join lateral flatten(metric_value) f2
 where
-  metric_key = 'spend_zero_mean_unit_variance';
+  metric_key = 'spend_zero_mean_unit_variance';