Profiler CLI #1623
Profiler CLI #1623
Conversation
sundarshankar89
changed the base branch from
main
to
feature/synapse_data_collector
|
✅ 19/19 passed, 1m20s total Running from acceptance #1134 |
labs.yml
Outdated
| @@ -67,8 +67,11 @@ commands: | |||
| - name: debug-me | |||
| description: "[INTERNAL] Debug SDK connectivity" | |||
| - name: install-assessment | |||
| description: "Install Assessment" | |||
| description: "[EXPERIMENTAL] Install Assessment" | |||
There was a problem hiding this comment.
What does Install assessment do?
There was a problem hiding this comment.
may be configure-assessment would be better word, it configures crednetials
There was a problem hiding this comment.
These credentials are the same we would use for Reconcile, no? Can we call it configure-source-credentials or something purposeful like that?
| logger.info( | ||
| f"Creating a virtual environment for Python script execution: ${venv_dir} for step: {step.name}" | ||
| ) |
There was a problem hiding this comment.
Stray $, maybe reword:
| logger.info( | |
| f"Creating a virtual environment for Python script execution: ${venv_dir} for step: {step.name}" | |
| ) | |
| logger.info(f"Creating a virtual environment for Python script execution, step {step.name}: {venv_dir}") |
| run([str(venv_pip), "install", "--upgrade", "pip"], check=True, stdout=DEVNULL, stderr=DEVNULL) | ||
| run([str(venv_pip), "install", *dependencies], check=True, stdout=DEVNULL, stderr=DEVNULL) |
There was a problem hiding this comment.
Before we drop the pip output, I think I'd like to see us passing these options:
--require-virtualenv(safety)--quiet(suppress normal output, but warning and higher will be emitted).--no-input(we're non-interactive at this point)--disable-pip-version-check(the check should be redundant, maybe its faster without)
In particular with --quiet I think maybe it's better to leave stdout and stderr alone so that we do get warnings, etc.
If this works I'll add the same to the transpiler side of things.
| from databricks.labs.remorph.assessments.profiler import Profiler | ||
|
|
||
|
|
||
| def test_supported_source_technologies(): |
There was a problem hiding this comment.
Remember to add -> None to the test declarations, otherwise the bodies aren't type-checked.
…ntry_point # Conflicts: # src/databricks/labs/lakebridge/assessments/profiler.py # src/databricks/labs/lakebridge/cli.py
|
✅ 14/14 passed, 1 skipped, 51s total Running from acceptance #1162 |
| @@ -61,3 +61,7 @@ commands: | |||
| default: null | |||
| - name: configure-reconcile | |||
| description: "Configure Necessary Reconcile Dependencies" | |||
| - name: execute-databse-profiler | |||
Changes
What does this PR do?
This introduces the intial version of CLI
Relevant implementation details
Caveats/things to watch out for when reviewing:
Linked issues
Resolves #..
Functionality
databricks labs remorph ...Tests