Skip to content

Importing a workflow with embedded tools fails if the latest CWL version is a development one #147

Closed
@nsoranzo

Description

@nsoranzo

Describe the bug
When importing a workflow via API, the workflow is updated to the latest CWL version (currently "v1.3.0-dev1", a development version) in WorkflowContentsManager.normalize_workflow_format() when workflow_proxy(workflow_path) is called.
Then, in the same method, each embedded tool is first persisted (representation = tool_reference_proxy.to_persistent_representation()) and then reloaded from the persisted representation in DynamicToolManager.create_tool() by calling tool_proxy(). This will try to update the tool again, but this will fail because it's already at a development version.

To Reproduce
Steps to reproduce the behavior:

  1. ./run_tests.sh -api lib/galaxy_test/api/test_workflows_cwl.py::TestCwlWorkflows::test_simplest_wf

Traceback:

  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 310, in create
    return self.__api_import_new_workflow(trans, payload, **kwd)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 622, in __api_import_new_workflow
    raw_workflow_description = self.__normalize_workflow(trans, data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 641, in __normalize_workflow
    return self.workflow_contents_manager.normalize_workflow_format(trans, as_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/managers/workflows.py", line 682, in normalize_workflow_format
    self.app.dynamic_tool_manager.create_tool(
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/managers/tools.py", line 105, in create_tool
    proxy = tool_proxy(
            ^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/parser.py", line 741, in tool_proxy
    return _to_cwl_tool_object(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/parser.py", line 811, in _to_cwl_tool_object
    cwl_tool = schema_loader.tool(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/schema.py", line 99, in tool
    process_definition = self.process_definition(raw_process_reference)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/schema.py", line 82, in process_definition
    loading_context, uri = resolve_and_validate_document(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.12/site-packages/cwltool/load_tool.py", line 540, in resolve_and_validate_document
    processobj = update.update(
                 ^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.12/site-packages/cwltool/update.py", line 357, in update
    (cdoc, version) = checkversion(doc, metadata, enable_dev)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.12/site-packages/cwltool/update.py", line 333, in checkversion
    raise ValidationException(
schema_salad.exceptions.ValidationException: Version 'v1.3.0-dev1' is a development or deprecated version.
 Update your document to a stable version (v1.0, v1.1, v1.2) or use --enable-dev to enable support for development and deprecated versions.

Potential solution
Instead of persisting and then reloading each embedded tool, create the dynamic tool from the existing tool_proxies available in the workflow proxy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions