pipen.scheduler
Provide builting schedulers
SchedulerPostInit
— Provides post init function for all schedulers</>LocalScheduler
— Local scheduler</>SgeScheduler
— SGE scheduler</>SlurmScheduler
— Slurm scheduler</>SshScheduler
— SSH scheduler</>GbatchScheduler
— Google Cloud Batch scheduler</>ContainerScheduler
— Scheduler to run jobs via containers (Docker/Podman/Apptainer)</>
get_scheduler
(
scheduler
)
(Type) — Get the scheduler by name of the scheduler class itself</>
pipen.scheduler.
SchedulerPostInit
(
)
Provides post init function for all schedulers
pipen.scheduler.
LocalScheduler
(
workdir
, forks=1
, error_strategy='ignore'
, num_retries=3
, prescript=''
, postscript=''
, jobname_prefix=None
, recheck_interval=600
, cwd=None
, **kwargs
)
Local scheduler
workdir
(str | Path | CloudPath) — The working directoryforks
(int, optional) — Max number of job forkserror_strategy
(str, optional) — The strategy when there is error happenednum_retries
(int, optional) — Max number of retries when error_strategy is retryprescript
(str, optional) — The prescript to run before the job commandIt is a piece of script that inserted into the wrapper script, running on the scheduler system.postscript
(str, optional) — The postscript to run when job finishedIt is a piece of script that inserted into the wrapper script, running on the scheduler system.jobname_prefix
(str | none, optional) — The prefix for the job namerecheck_interval
(int, optional) — The interval to recheck the job status.Default is every 600 polls (each takes about 0.1 seconds).cwd
(str | Path, optional) — The working directory for the job command wrapper**kwargs
— Other arguments for the scheduler
job_class
— The job classjobcmd_wrapper_init
— The init script for the job command wrapperjobcmd_wrapper_init
(str) — The init script for the job command wrapper</>name
— The name of the scheduler
create_job
(
index
,cmd
,envs
)
(Job) — Create a job</>job_is_running
(
job
)
(bool) — Tell if a job is really running, not only the job.jid_file</>job_is_submitted_or_running
(
job
)
(bool) — Check if a job is already submitted or running</>jobcmd_end
(
job
)
(str) — The job command end</>jobcmd_init
(
job
)
(str) — The job command init</>jobcmd_prep
(
job
)
(str) — The job command preparation</>jobcmd_shebang
(
job
)
(str) — The shebang of the wrapper script</>kill_job
(
job
)
— Kill a job asynchronously</>kill_job_and_update_status
(
job
)
— Kill a job and update its status</>kill_running_jobs
(
jobs
)
— Try to kill all running jobs</>polling_jobs
(
jobs
,on
,polling_counter
)
(bool) — Check if all jobs are done or new jobs can submit</>retry_job
(
job
)
— Retry a job</>submit_job
(
job
,_mounted
)
(int) — Submit a job locally</>submit_job_and_update_status
(
job
)
— Submit and update the status</>wrap_job_script
(
job
)
(str) — Wrap the job script</>wrapped_job_script
(
job
)
(SpecPath) — Get the wrapped job script</>
create_job
(
index
, cmd
, envs=None
)
Create a job
index
(int) — The index of the jobcmd
(Union) — The command of the job
The job
submit_job_and_update_status
(
job
)
Submit and update the status
- Check if the job is already submitted or running
- If not, run the hook
- If the hook is not cancelled, clean the job
- Submit the job, raising an exception if it fails
- If the job is submitted successfully, update the status
- If the job fails to submit, update the status and write stderr to the job file
job
(Job) — The job
retry_job
(
job
)
Retry a job
job
(Job) — The job
kill_job_and_update_status
(
job
)
Kill a job and update its status
job
(Job) — The job
polling_jobs
(
jobs
, on
, polling_counter
)
Check if all jobs are done or new jobs can submit
jobs
(List) — The list of jobson
(str) — query on status:submittable
orall_done
polling_counter
(int) — The polling counter, used to limit the number of polls orskip some polls if the scheduler is busy.
True if yes otherwise False.
kill_running_jobs
(
jobs
)
Try to kill all running jobs
jobs
(List) — The list of jobs
job_is_submitted_or_running
(
job
)
Check if a job is already submitted or running
job
(Job) — The job
True if yes otherwise False.
jobcmd_shebang
(
job
)
→ str
The shebang of the wrapper script
jobcmd_init
(
job
)
→ str
The job command init
jobcmd_prep
(
job
)
→ str
The job command preparation
jobcmd_end
(
job
)
→ str
The job command end
wrap_job_script
(
job
)
Wrap the job script
job
(Job) — The job
The wrapped script
wrapped_job_script
(
job
)
Get the wrapped job script
job
(Job) — The job
The path of the wrapped job script
submit_job
(
job
, _mounted=False
)
Submit a job locally
job
(Job) — The job_mounted
(bool, optional) — Whether to use the mounted path of the wrapped job scriptUsed internally for container scheduler
The process id
kill_job
(
job
)
Kill a job asynchronously
job
(Job) — The job
job_is_running
(
job
)
Tell if a job is really running, not only the job.jid_file
In case where the jid file is not cleaned when job is done.
job
(Job) — The job
True if it is, otherwise False
pipen.scheduler.
SgeScheduler
(
*args
, **kwargs
)
SGE scheduler
**kwargs
— Other arguments for the scheduler
job_class
— The job classjobcmd_wrapper_init
— The init script for the job command wrapperjobcmd_wrapper_init
(str) — The init script for the job command wrapper</>name
— The name of the scheduler
create_job
(
index
,cmd
,envs
)
(Job) — Create a job</>job_is_running
(
job
)
(bool) — Tell if a job is really running, not only the job.jid_file</>job_is_submitted_or_running
(
job
)
(bool) — Check if a job is already submitted or running</>jobcmd_end
(
job
)
(str) — The job command end</>jobcmd_init
(
job
)
(str) — The job command init</>jobcmd_prep
(
job
)
(str) — The job command preparation</>jobcmd_shebang
(
job
)
(str) — The shebang of the wrapper script</>kill_job
(
job
)
— Kill a job on SGE</>kill_job_and_update_status
(
job
)
— Kill a job and update its status</>kill_running_jobs
(
jobs
)
— Try to kill all running jobs</>polling_jobs
(
jobs
,on
,polling_counter
)
(bool) — Check if all jobs are done or new jobs can submit</>retry_job
(
job
)
— Retry a job</>submit_job
(
job
)
(str) — Submit a job to SGE</>submit_job_and_update_status
(
job
)
— Submit and update the status</>wrap_job_script
(
job
)
(str) — Wrap the job script</>wrapped_job_script
(
job
)
(SpecPath) — Get the wrapped job script</>
create_job
(
index
, cmd
, envs=None
)
Create a job
index
(int) — The index of the jobcmd
(Union) — The command of the job
The job
submit_job_and_update_status
(
job
)
Submit and update the status
- Check if the job is already submitted or running
- If not, run the hook
- If the hook is not cancelled, clean the job
- Submit the job, raising an exception if it fails
- If the job is submitted successfully, update the status
- If the job fails to submit, update the status and write stderr to the job file
job
(Job) — The job
retry_job
(
job
)
Retry a job
job
(Job) — The job
kill_job_and_update_status
(
job
)
Kill a job and update its status
job
(Job) — The job
polling_jobs
(
jobs
, on
, polling_counter
)
Check if all jobs are done or new jobs can submit
jobs
(List) — The list of jobson
(str) — query on status:submittable
orall_done
polling_counter
(int) — The polling counter, used to limit the number of polls orskip some polls if the scheduler is busy.
True if yes otherwise False.
kill_running_jobs
(
jobs
)
Try to kill all running jobs
jobs
(List) — The list of jobs
job_is_submitted_or_running
(
job
)
Check if a job is already submitted or running
job
(Job) — The job
True if yes otherwise False.
jobcmd_init
(
job
)
→ str
The job command init
jobcmd_prep
(
job
)
→ str
The job command preparation
jobcmd_end
(
job
)
→ str
The job command end
wrap_job_script
(
job
)
Wrap the job script
job
(Job) — The job
The wrapped script
wrapped_job_script
(
job
)
Get the wrapped job script
job
(Job) — The job
The path of the wrapped job script
jobcmd_shebang
(
job
)
→ str
The shebang of the wrapper script
submit_job
(
job
)
Submit a job to SGE
job
(Job) — The job
The job id
kill_job
(
job
)
Kill a job on SGE
job
(Job) — The job
job_is_running
(
job
)
Tell if a job is really running, not only the job.jid_file
In case where the jid file is not cleaned when job is done.
job
(Job) — The job
True if it is, otherwise False
pipen.scheduler.
SlurmScheduler
(
*args
, **kwargs
)
Slurm scheduler
**kwargs
— Other arguments for the scheduler
job_class
— The job classjobcmd_wrapper_init
— The init script for the job command wrapperjobcmd_wrapper_init
(str) — The init script for the job command wrapper</>name
— The name of the scheduler
create_job
(
index
,cmd
,envs
)
(Job) — Create a job</>job_is_running
(
job
)
(bool) — Tell if a job is really running, not only the job.jid_file</>job_is_submitted_or_running
(
job
)
(bool) — Check if a job is already submitted or running</>jobcmd_end
(
job
)
(str) — The job command end</>jobcmd_init
(
job
)
(str) — The job command init</>jobcmd_prep
(
job
)
(str) — The job command preparation</>jobcmd_shebang
(
job
)
(str) — The shebang of the wrapper script</>kill_job
(
job
)
— Kill a job on Slurm</>kill_job_and_update_status
(
job
)
— Kill a job and update its status</>kill_running_jobs
(
jobs
)
— Try to kill all running jobs</>polling_jobs
(
jobs
,on
,polling_counter
)
(bool) — Check if all jobs are done or new jobs can submit</>retry_job
(
job
)
— Retry a job</>submit_job
(
job
)
(str) — Submit a job to Slurm</>submit_job_and_update_status
(
job
)
— Submit and update the status</>wrap_job_script
(
job
)
(str) — Wrap the job script</>wrapped_job_script
(
job
)
(SpecPath) — Get the wrapped job script</>
create_job
(
index
, cmd
, envs=None
)
Create a job
index
(int) — The index of the jobcmd
(Union) — The command of the job
The job
submit_job_and_update_status
(
job
)
Submit and update the status
- Check if the job is already submitted or running
- If not, run the hook
- If the hook is not cancelled, clean the job
- Submit the job, raising an exception if it fails
- If the job is submitted successfully, update the status
- If the job fails to submit, update the status and write stderr to the job file
job
(Job) — The job
retry_job
(
job
)
Retry a job
job
(Job) — The job
kill_job_and_update_status
(
job
)
Kill a job and update its status
job
(Job) — The job
polling_jobs
(
jobs
, on
, polling_counter
)
Check if all jobs are done or new jobs can submit
jobs
(List) — The list of jobson
(str) — query on status:submittable
orall_done
polling_counter
(int) — The polling counter, used to limit the number of polls orskip some polls if the scheduler is busy.
True if yes otherwise False.
kill_running_jobs
(
jobs
)
Try to kill all running jobs
jobs
(List) — The list of jobs
job_is_submitted_or_running
(
job
)
Check if a job is already submitted or running
job
(Job) — The job
True if yes otherwise False.
jobcmd_init
(
job
)
→ str
The job command init
jobcmd_prep
(
job
)
→ str
The job command preparation
jobcmd_end
(
job
)
→ str
The job command end
wrap_job_script
(
job
)
Wrap the job script
job
(Job) — The job
The wrapped script
wrapped_job_script
(
job
)
Get the wrapped job script
job
(Job) — The job
The path of the wrapped job script
jobcmd_shebang
(
job
)
→ str
The shebang of the wrapper script
submit_job
(
job
)
Submit a job to Slurm
job
(Job) — The job
The job id
kill_job
(
job
)
Kill a job on Slurm
job
(Job) — The job
job_is_running
(
job
)
Tell if a job is really running, not only the job.jid_file
In case where the jid file is not cleaned when job is done.
job
(Job) — The job
True if it is, otherwise False
pipen.scheduler.
SshScheduler
(
*args
, **kwargs
)
SSH scheduler
**kwargs
— Other arguments for the scheduler
job_class
— The job classjobcmd_wrapper_init
— The init script for the job command wrapperjobcmd_wrapper_init
(str) — The init script for the job command wrapper</>name
— The name of the scheduler
create_job
(
index
,cmd
,envs
)
(Job) — Create a job</>job_is_running
(
job
)
(bool) — Tell if a job is really running, not only the job.jid_file</>job_is_submitted_or_running
(
job
)
(bool) — Check if a job is already submitted or running</>jobcmd_end
(
job
)
(str) — The job command end</>jobcmd_init
(
job
)
(str) — The job command init</>jobcmd_prep
(
job
)
(str) — The job command preparation</>jobcmd_shebang
(
job
)
(str) — The shebang of the wrapper script</>kill_job
(
job
)
— Kill a job on SSH</>kill_job_and_update_status
(
job
)
— Kill a job and update its status</>kill_running_jobs
(
jobs
)
— Try to kill all running jobs</>polling_jobs
(
jobs
,on
,polling_counter
)
(bool) — Check if all jobs are done or new jobs can submit</>retry_job
(
job
)
— Retry a job</>submit_job
(
job
)
(str) — Submit a job to SSH</>submit_job_and_update_status
(
job
)
— Submit and update the status</>wrap_job_script
(
job
)
(str) — Wrap the job script</>wrapped_job_script
(
job
)
(SpecPath) — Get the wrapped job script</>
create_job
(
index
, cmd
, envs=None
)
Create a job
index
(int) — The index of the jobcmd
(Union) — The command of the job
The job
submit_job_and_update_status
(
job
)
Submit and update the status
- Check if the job is already submitted or running
- If not, run the hook
- If the hook is not cancelled, clean the job
- Submit the job, raising an exception if it fails
- If the job is submitted successfully, update the status
- If the job fails to submit, update the status and write stderr to the job file
job
(Job) — The job
retry_job
(
job
)
Retry a job
job
(Job) — The job
kill_job_and_update_status
(
job
)
Kill a job and update its status
job
(Job) — The job
polling_jobs
(
jobs
, on
, polling_counter
)
Check if all jobs are done or new jobs can submit
jobs
(List) — The list of jobson
(str) — query on status:submittable
orall_done
polling_counter
(int) — The polling counter, used to limit the number of polls orskip some polls if the scheduler is busy.
True if yes otherwise False.
kill_running_jobs
(
jobs
)
Try to kill all running jobs
jobs
(List) — The list of jobs
job_is_submitted_or_running
(
job
)
Check if a job is already submitted or running
job
(Job) — The job
True if yes otherwise False.
jobcmd_shebang
(
job
)
→ str
The shebang of the wrapper script
jobcmd_init
(
job
)
→ str
The job command init
jobcmd_prep
(
job
)
→ str
The job command preparation
jobcmd_end
(
job
)
→ str
The job command end
wrap_job_script
(
job
)
Wrap the job script
job
(Job) — The job
The wrapped script
wrapped_job_script
(
job
)
Get the wrapped job script
job
(Job) — The job
The path of the wrapped job script
submit_job
(
job
)
Submit a job to SSH
job
(Job) — The job
The job id
kill_job
(
job
)
Kill a job on SSH
job
(Job) — The job
job_is_running
(
job
)
Tell if a job is really running, not only the job.jid_file
In case where the jid file is not cleaned when job is done.
job
(Job) — The job
True if it is, otherwise False
pipen.scheduler.
GbatchScheduler
(
*args
, project
, location
, mount=None
, service_account=None
, network=None
, subnetwork=None
, no_external_ip_address=None
, machine_type=None
, provisioning_model=None
, image_uri=None
, entrypoint=None
, commands=None
, runnables=None
, **kwargs
)
Google Cloud Batch scheduler
**kwargs
— Keyword arguments for the configuration of a job (e.g. taskGroups).See more details at https://cloud.google.com/batch/docs/get-started.*args
— Positional arguments for the base classproject
(str) — Google Cloud project IDlocation
(str) — Google Cloud region or zonemount
(str | Sequence[str] | None, optional) — GCS path to mount (e.g. gs://my-bucket:/mnt/my-bucket)You can pass a list of mounts.service_account
(str | none, optional) — GCP service account email (e.g. test-account@example.com)network
(str | none, optional) — GCP network (e.g. default-network)subnetwork
(str | none, optional) — GCP subnetwork (e.g. regions/us-central1/subnetworks/default)no_external_ip_address
(bool | none, optional) — Whether to disable external IP addressmachine_type
(str | none, optional) — GCP machine type (e.g. e2-standard-4)provisioning_model
(str | none, optional) — GCP provisioning model (e.g. SPOT)image_uri
(str | none, optional) — Container image URI (e.g. ubuntu-2004-lts)entrypoint
(str, optional) — Container entrypoint (e.g. /bin/bash)commands
(str | Sequence[str] | None, optional) — The command list to run in the container.There are three ways to specify the commands: 1. If no entrypoint is specified, the final command will be [commands, wrapped_script], where the entrypoint is the wrapper script interpreter that is determined byJOBCMD_WRAPPER_LANG
(e.g. /bin/bash), commands is the list you provided, and wrapped_script is the path to the wrapped job script. 2. You can specify something like "-c", then the final command will be ["-c", "wrapper_script_interpreter, wrapper_script"] 3. You can use the placeholders{lang}
and{script}
in the commands list, where{lang}
will be replaced with the interpreter (e.g. /bin/bash) and{script}
will be replaced with the path to the wrapped job script. For example, you can specify ["{lang} {script}"] and the final command will be ["wrapper_interpreter, wrapper_script"]runnables
(Sequence[dict] | None, optional) — Additional runnables to run before or after the main job.Each runnable should be a dictionary that follows the GCP Batch API specification. You can also specify an "order" key in the dictionary to control the execution order of the runnables. Runnables with negative order will be executed before the main job, and those with non-negative order will be executed after the main job. The main job runnable will always be executed in the order it is defined in the list.
jobcmd_wrapper_init
— The init script for the job command wrapperjobcmd_wrapper_init
(str) — The init script for the job command wrapper</>name
— The name of the scheduler
create_job
(
index
,cmd
,envs
)
(Job) — Create a job</>job_is_running
(
job
)
(bool) — Check if a job is really running</>job_is_submitted_or_running
(
job
)
(bool) — Check if a job is already submitted or running</>jobcmd_end
(
job
)
(str) — The job command end</>jobcmd_init
(
job
)
(str) — The job command init</>jobcmd_prep
(
job
)
(str) — The job command preparation</>jobcmd_shebang
(
job
)
(str) — The shebang of the wrapper script</>kill_job
(
job
)
— Kill a job</>kill_job_and_update_status
(
job
)
— Kill a job and update its status</>kill_running_jobs
(
jobs
)
— Try to kill all running jobs</>polling_jobs
(
jobs
,on
,polling_counter
)
(bool) — Check if all jobs are done or new jobs can submit</>retry_job
(
job
)
— Retry a job</>submit_job
(
job
)
(str) — Submit a job</>submit_job_and_update_status
(
job
)
— Submit and update the status</>wrap_job_script
(
job
)
(str) — Wrap the job script</>wrapped_job_script
(
job
)
(SpecPath) — Get the wrapped job script</>
create_job
(
index
, cmd
, envs=None
)
Create a job
index
(int) — The index of the jobcmd
(Union) — The command of the job
The job
submit_job_and_update_status
(
job
)
Submit and update the status
- Check if the job is already submitted or running
- If not, run the hook
- If the hook is not cancelled, clean the job
- Submit the job, raising an exception if it fails
- If the job is submitted successfully, update the status
- If the job fails to submit, update the status and write stderr to the job file
job
(Job) — The job
retry_job
(
job
)
Retry a job
job
(Job) — The job
kill_job_and_update_status
(
job
)
Kill a job and update its status
job
(Job) — The job
polling_jobs
(
jobs
, on
, polling_counter
)
Check if all jobs are done or new jobs can submit
jobs
(List) — The list of jobson
(str) — query on status:submittable
orall_done
polling_counter
(int) — The polling counter, used to limit the number of polls orskip some polls if the scheduler is busy.
True if yes otherwise False.
kill_running_jobs
(
jobs
)
Try to kill all running jobs
jobs
(List) — The list of jobs
job_is_submitted_or_running
(
job
)
Check if a job is already submitted or running
job
(Job) — The job
True if yes otherwise False.
jobcmd_shebang
(
job
)
→ str
The shebang of the wrapper script
jobcmd_prep
(
job
)
→ str
The job command preparation
jobcmd_end
(
job
)
→ str
The job command end
wrap_job_script
(
job
)
Wrap the job script
job
(Job) — The job
The wrapped script
wrapped_job_script
(
job
)
Get the wrapped job script
job
(Job) — The job
The path of the wrapped job script
submit_job
(
job
)
Submit a job
job
(Job) — The job
The unique id in the scheduler system
kill_job
(
job
)
Kill a job
job
(Job) — The job
job_is_running
(
job
)
Check if a job is really running
job
(Job) — The job
True if yes otherwise False.
jobcmd_init
(
job
)
→ str
The job command init
pipen.scheduler.
ContainerScheduler
(
image
, entrypoint='/bin/bash'
, bin='docker'
, volumes=None
, remove=True
, user=None
, bin_args=None
, **kwargs
)
Scheduler to run jobs via containers (Docker/Podman/Apptainer)
**kwargs
— Additional arguments passed to parent Schedulerimage
(str) — Container image to use for running jobsentrypoint
(str | List[str], optional) — Entrypoint command for the containerbin
(str, optional) — Path to container runtime binary (e.g. /path/to/docker)volumes
(str | Sequence[str] | None, optional) — host:container volume mapping string or stringsor named volume mapping likeMOUNTED=/path/on/host
then it will be mounted to/mnt/disks/MOUNTED
in the container. You can use environment variableMOUNTED
in your job scripts to refer to the mounted path.remove
(bool, optional) — Whether to remove the container after execution.Only applies to Docker/Podman.user
(str | none, optional) — User to run the container as (only for Docker/Podman)By default, it runs as the current user (os.getuid() and os.getgid())bin_args
(List[str] | None, optional) — Additional arguments to pass to the container runtime
job_class
— The job classjobcmd_wrapper_init
— The init script for the job command wrapperjobcmd_wrapper_init
(str) — The init script for the job command wrapper</>name
— The name of the scheduler
create_job
(
index
,cmd
,envs
)
(Job) — Create a job</>job_is_running
(
job
)
(bool) — Tell if a job is really running, not only the job.jid_file</>job_is_submitted_or_running
(
job
)
(bool) — Check if a job is already submitted or running</>jobcmd_end
(
job
)
(str) — The job command end</>jobcmd_init
(
job
)
(str) — The job command init</>jobcmd_prep
(
job
)
(str) — The job command preparation</>jobcmd_shebang
(
job
)
(str) — The shebang of the wrapper script</>kill_job
(
job
)
— Kill a job asynchronously</>kill_job_and_update_status
(
job
)
— Kill a job and update its status</>kill_running_jobs
(
jobs
)
— Try to kill all running jobs</>polling_jobs
(
jobs
,on
,polling_counter
)
(bool) — Check if all jobs are done or new jobs can submit</>retry_job
(
job
)
— Retry a job</>submit_job
(
job
)
(int) — Submit a job locally</>submit_job_and_update_status
(
job
)
— Submit and update the status</>wrap_job_script
(
job
)
(str) — Wrap the job script</>wrapped_job_script
(
job
)
(SpecPath) — Get the wrapped job script</>
create_job
(
index
, cmd
, envs=None
)
Create a job
index
(int) — The index of the jobcmd
(Union) — The command of the job
The job
submit_job_and_update_status
(
job
)
Submit and update the status
- Check if the job is already submitted or running
- If not, run the hook
- If the hook is not cancelled, clean the job
- Submit the job, raising an exception if it fails
- If the job is submitted successfully, update the status
- If the job fails to submit, update the status and write stderr to the job file
job
(Job) — The job
retry_job
(
job
)
Retry a job
job
(Job) — The job
kill_job_and_update_status
(
job
)
Kill a job and update its status
job
(Job) — The job
polling_jobs
(
jobs
, on
, polling_counter
)
Check if all jobs are done or new jobs can submit
jobs
(List) — The list of jobson
(str) — query on status:submittable
orall_done
polling_counter
(int) — The polling counter, used to limit the number of polls orskip some polls if the scheduler is busy.
True if yes otherwise False.
kill_running_jobs
(
jobs
)
Try to kill all running jobs
jobs
(List) — The list of jobs
job_is_submitted_or_running
(
job
)
Check if a job is already submitted or running
job
(Job) — The job
True if yes otherwise False.
jobcmd_prep
(
job
)
→ str
The job command preparation
jobcmd_end
(
job
)
→ str
The job command end
wrap_job_script
(
job
)
Wrap the job script
job
(Job) — The job
The wrapped script
kill_job
(
job
)
Kill a job asynchronously
job
(Job) — The job
job_is_running
(
job
)
Tell if a job is really running, not only the job.jid_file
In case where the jid file is not cleaned when job is done.
job
(Job) — The job
True if it is, otherwise False
wrapped_job_script
(
job
)
Get the wrapped job script
job
(Job) — The job
The path of the wrapped job script
jobcmd_shebang
(
job
)
→ str
The shebang of the wrapper script
submit_job
(
job
)
Submit a job locally
job
(Job) — The job
The process id
jobcmd_init
(
job
)
→ str
The job command init
pipen.scheduler.
get_scheduler
(
scheduler
)
Get the scheduler by name of the scheduler class itself
scheduler
(Union) — The scheduler class or name
The scheduler class