[BUG]: Datastage transpile failing

[Prakash0108]

Is there an existing issue for this?

• I have searched the existing issues

Category of Bug / Issue

Converter bug

Current Behavior

Datastage transpile failing without any clear explanation.

error.log

input-xml.zip

Expected Behavior

It should succeed without failing and expecting the correct output

Steps To Reproduce

commands used:

databricks labs lakebridge analyze --source-directory /workspaces/lakebridge/datastage/input-xml --report-file /workspaces/lakebridge/datastage/analysis/datastage-inventory.xlsx

databricks labs lakebridge transpile --source-dialect datastage --input-source /workspaces/lakebridge/datastage/input-xml --output-folder /workspaces/lakebridge/datastage/output --error-file-path /workspaces/lakebridge/datastage/error/error.log

Relevant log output or Exception details

TranspileError(code=CONVERSION-FAILURE, kind=INTERNAL, severity=ERROR, path='/workspaces/codespaces-blank/lakebridge/datastage/input/sample1_job_Version3.xml', message='Error transpiling file: sample1_job_Version3.xml:

Sample Query

Operating System

Windows

Version

latest via Databricks CLI

[sundarshankar89]

@Prakash0108 can you share the console log after adding --debug to the command line.

And also did you run databricks labs lakebridge install-transpile before running transpile ?

Finally if you still have errors, some additional log which will help should be present inside $HOME/.databricks/labs/remorph-transpilers/bladebridge/lib/lsp-server.log

[Prakash0108]

yes I ran databricks labs lakebridge install-transpile before running transpile

Pasted the console log below:
@Prakash0108 ➜ /workspaces/lakebridge (codespace-animated-space-bassoon-q747j9xvv65jf69jq) $ databricks labs lakebridge transpile --source-dialect datastage --input-source /workspaces/codespaces-blank/lakebridge/datastage/input-xml --output-folder /workspaces/codespaces-blank/lakebridge/datastage/output --debug
14:52:12 Info: start pid=3425 version=0.258.0 args="databricks, labs, lakebridge, transpile, --source-dialect, datastage, --input-source, /workspaces/codespaces-blank/lakebridge/datastage/input-xml, --output-folder, /workspaces/codespaces-blank/lakebridge/datastage/output, --debug"
14:52:12 Debug: Loading installed version info from: /home/codespace/.databricks/labs/lakebridge/state/version.json pid=3425
14:52:12 Debug: Loading login configuration from: /home/codespace/.databricks/labs/lakebridge/config/login.json pid=3425
14:52:12 Debug: Using workspace-level login profile: DEFAULT pid=3425
14:52:12 Debug: Loading DEFAULT profile from /home/codespace/.databrickscfg pid=3425 sdk=true
14:52:12 Debug: Resolved login: Config: host=https://dbc-2eaa663a-e521.cloud.databricks.com, token=***, profile=DEFAULT, config_file=/home/codespace/.databrickscfg pid=3425 sdk=true
14:52:12 Debug: Passing down environment variables: DATABRICKS_HOST, DATABRICKS_TOKEN pid=3425
14:52:12 Debug: Forwarding subprocess: /home/codespace/.databricks/labs/lakebridge/state/venv/bin/python3 /home/codespace/.databricks/labs/lakebridge/lib/src/databricks/labs/lakebridge/cli.py {"command":"transpile","flags":{"catalog-name":"","error-file-path":"","input-source":"/workspaces/codespaces-blank/lakebridge/datastage/input-xml","log_level":"debug","output-folder":"/workspaces/codespaces-blank/lakebridge/datastage/output","schema-name":"","skip-validation":"true","source-dialect":"datastage","transpiler-config-path":""},"output_type":""} pid=3425
14:52:12 Debug: starting: /home/codespace/.databricks/labs/lakebridge/state/venv/bin/python3 /home/codespace/.databricks/labs/lakebridge/lib/src/databricks/labs/lakebridge/cli.py {"command":"transpile","flags":{"catalog-name":"","error-file-path":"","input-source":"/workspaces/codespaces-blank/lakebridge/datastage/input-xml","log_level":"debug","output-folder":"/workspaces/codespaces-blank/lakebridge/datastage/output","schema-name":"","skip-validation":"true","source-dialect":"datastage","transpiler-config-path":""},"output_type":""} pid=3425
14:52:14 DEBUG [d.labs.lakebridge] Leaving DATABRICKS_HOST as-is: https://dbc-2eaa663a-e521.cloud.databricks.com
14:52:14 DEBUG [databricks.sdk] Loaded from environment
14:52:14 DEBUG [databricks.sdk] Attempting to configure auth: pat
14:52:15 DEBUG [databricks.sdk] GET /api/2.0/preview/scim/v2/Me
< 200 OK
< {
< "active": true,
< "displayName": "Senthil Kumarr Malli Sudarsan",
< "emails": [
< {
< "primary": true,
< "type": "work",
< "value": "REDACTED"
< }
< ],
< "entitlements": [
< {
< "value": "REDACTED"
< },
< "... (1 additional elements)"
< ],
< "groups": [
< {
< "$ref": "Groups/81652439326230",
< "display": "admins",
< "type": "direct",
< "value": "REDACTED"
< },
< "... (1 additional elements)"
< ],
< "id": "4815752442862392",
< "name": {
< "familyName": "Sudarsan",
< "givenName": "Senthil Kumarr Malli"
< },
< "schemas": [
< "urn:ietf:params:scim:schemas:core:2.0:User",
< "... (1 additional elements)"
< ],
< "userName": "senthilkumarr.ma@tigeranalytics.com"
< }
14:52:15 DEBUG [d.l.blueprint.installation] Loading TranspileConfig from config.yml
14:52:16 DEBUG [databricks.sdk] GET /api/2.0/workspace/export?path=/Users/senthilkumarr.ma@tigeranalytics.com/.lakebridge/config.yml&direct_download=true
< 200 OK
< [raw stream]
14:52:16 DEBUG [d.labs.lakebridge] Preconfigured transpiler config: TranspileConfig(transpiler_config_path='/home/codespace/.databricks/labs/remorph-transpilers/bladebridge/lib/config.yml', source_dialect='datastage', input_source=None, output_folder='/workspaces/codespaces-blank/transpiled', error_file_path='/workspaces/codespaces-blank/errors.log', sdk_config=None, skip_validation=True, catalog_name='remorph', schema_name='transpiler', transpiler_options={'overrides-file': None, 'target-tech': 'PYSPARK'})
14:52:16 DEBUG [d.sdk.useragent] Adding cmd/execute-transpile to User-Agent
14:52:16 DEBUG [d.labs.lakebridge] Pending source_dialect override: 'datastage'
14:52:16 DEBUG [d.labs.lakebridge] Setting input_source to: '/workspaces/codespaces-blank/lakebridge/datastage/input-xml'
14:52:16 ERROR [d.l.lakebridge.transpile] Failed to call transpile
Traceback (most recent call last):
File "/home/codespace/.databricks/labs/lakebridge/state/venv/lib/python3.12/site-packages/databricks/labs/blueprint/cli.py", line 118, in _route
cmd.fn(**kwargs)
File "/home/codespace/.databricks/labs/lakebridge/lib/src/databricks/labs/lakebridge/cli.py", line 125, in transpile
checker.use_input_source(input_source)
File "/home/codespace/.databricks/labs/lakebridge/lib/src/databricks/labs/lakebridge/cli.py", line 239, in use_input_source
self._validate_input_source(
File "/home/codespace/.databricks/labs/lakebridge/lib/src/databricks/labs/lakebridge/cli.py", line 234, in _validate_input_source
raise_validation_exception(msg)
File "/home/codespace/.databricks/labs/lakebridge/lib/src/databricks/labs/lakebridge/cli.py", line 54, in raise_validation_exception
raise ValueError(msg)