Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing a workflow with embedded tools fails if the latest CWL version is a development one #147

Open
nsoranzo opened this issue Oct 1, 2024 · 0 comments

Comments

@nsoranzo
Copy link
Collaborator

nsoranzo commented Oct 1, 2024

Describe the bug
When importing a workflow via API, the workflow is updated to the latest CWL version (currently "v1.3.0-dev1", a development version) in WorkflowContentsManager.normalize_workflow_format() when workflow_proxy(workflow_path) is called.
Then, in the same method, each embedded tool is first persisted (representation = tool_reference_proxy.to_persistent_representation()) and then reloaded from the persisted representation in DynamicToolManager.create_tool() by calling tool_proxy(). This will try to update the tool again, but this will fail because it's already at a development version.

To Reproduce
Steps to reproduce the behavior:

  1. ./run_tests.sh -api lib/galaxy_test/api/test_workflows_cwl.py::TestCwlWorkflows::test_simplest_wf

Traceback:

  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 317, in create
    return self.__api_import_new_workflow(trans, payload, **kwd)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 627, in __api_import_new_workflow
    raw_workflow_description = self.__normalize_workflow(trans, data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 646, in __normalize_workflow
    return self.workflow_contents_manager.normalize_workflow_format(trans, as_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/managers/workflows.py", line 675, in normalize_workflow_format
        elif workflow_class == "Workflow":
            from galaxy.tool_util.cwl import workflow_proxy

            # create a temporary file for the workflow if it is provided
            # as JSON, to make it parseable by the WorkflowProxy
            if workflow_path is None:
                with tempfile.NamedTemporaryFile(mode="w+", delete=False) as f:
                    json.dump(as_dict, f)
                    workflow_path = f.name
                if object_id:
                    workflow_path += "#" + object_id
                wf_proxy = workflow_proxy(workflow_path)
                os.unlink(f.name)
            else:
                # TODO: consume and use object_id...
                if object_id:
                    workflow_path += "#" + object_id
                wf_proxy = workflow_proxy(workflow_path)
            tool_reference_proxies = wf_proxy.tool_reference_proxies()
            for tool_reference_proxy in tool_reference_proxies:
                # TODO: Namespace IDS in workflows.
                representation = tool_reference_proxy.to_persistent_representation()
                self.app.dynamic_tool_manager.create_tool(
                    trans,
                    {
                        "uuid": tool_reference_proxy.uuid,
                        "representation": representation,
                    },
                    allow_load=True,
                )
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/managers/tools.py", line 105, in create_tool
                else:
                    # Build a tool proxy so that we can convert to the persistable
                    # hash.
                    proxy = tool_proxy(
                        tool_object=representation["raw_process_reference"],
                        tool_directory=tool_directory,
                        uuid=uuid,
                    )
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/parser.py", line 742, in tool_proxy
    return _to_cwl_tool_object(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/parser.py", line 812, in _to_cwl_tool_object
    cwl_tool = schema_loader.tool(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/schema.py", line 101, in tool
    process_definition = self.process_definition(raw_process_reference)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/tool_util/cwl/schema.py", line 83, in process_definition
    loading_context, uri = resolve_and_validate_document(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.12/site-packages/cwltool/load_tool.py", line 549, in resolve_and_validate_document
    processobj = update.update(
                 ^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.12/site-packages/cwltool/update.py", line 358, in update
    (cdoc, version) = checkversion(doc, metadata, enable_dev)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/.venv/lib/python3.12/site-packages/cwltool/update.py", line 334, in checkversion
    raise ValidationException(
schema_salad.exceptions.ValidationException: Version 'v1.3.0-dev1' is a development or deprecated version.

Potential solution
Instead of persisting and then reloading each embedded tool, create the dynamic tool from the existing tool_proxies available in the workflow proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant