Open
Description
What happened?
Batch now supports organizing jobs into a hierarchy through job groups, but this is so far only used internally by Query on Batch. Users who want to leverage this structure cannot do so through the hailtop.batch
interface. All jobs created by e.g. b.new_job()
are implicitly assigned to the "root" job group. We should add the following to hailtop.batch
:
- A
JobGroup
class to the public interface that shares any relevant methods that currently exist on theBatch
class likenew_{bash|python}_job
,wait
, etc. - A method on
Batch
to create a new job group and a method onJobGroup
to create a child job group - A
JobGroup
should not have arun
method, because there is no sound way in the lower-level batch client currently to only submit a subset of staged jobs/groups and it's unclear whether that behavior would ever be desired.
Version
0.2.132
Relevant log output
No response