Skip to content

[batch] Resolve not being able to wait for a batch to finish when called within a job of that batch #14338

Open
@jigold

Description

@jigold

What happened?

Removing the FIXME messages in hailtop.batch_client.aioclient in favor of a proper issue. This error needs to be fixed for hailtop.batch_client.aioclient.JobGroup as well once that code merges with #14282.

    # FIXME Error if this is called while within a job of the same Batch
    async def wait(
        self,
        *,
        disable_progress_bar: bool = False,
        description: str = '',
        progress: Optional[BatchProgressBar] = None,
        starting_job: int = 1,
    ) -> Dict[str, Any]:
        self._raise_if_not_created()
        if description:
            description += ': '
        if progress is not None:
            return await self._wait(description, progress, disable_progress_bar, starting_job)
        with BatchProgressBar(disable=disable_progress_bar) as progress2:
            return await self._wait(description, progress2, disable_progress_bar, starting_job)

I'm not sure what the best fix is for this. An advanced user who wants to use Batch inside of Batch should really be making separate job groups from the job that job group is in and waiting for the job group that job is not a part of.

Version

0.2.128

Relevant log output

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions