Skip to content

pipen.job

module

pipen.job

</>

Provide the Job class

Classes
  • Job — The job for pipen</>
class

pipen.job.Job(*args, **kwargs)

</>
Bases
xqute.job.Job abc.ABC pipen._job_caching.JobCaching

The job for pipen

Attributes
  • CMD_WRAPPER_SHELL — The shell to run the wrapped script
  • CMD_WRAPPER_TEMPLATE — The template for job wrapping
  • _error_retry — Whether we should retry if error happened
  • _num_retries — Total number of retries
  • _rc — The return code of the job
  • _status — The status of the job
  • _wrapped_cmd — The wrapped cmd, used for job submission
  • cached — Check if a job is cached</>
  • cmd — The command
  • hook_done — Mark whether hooks have already been. Since we don't havea trigger for job finished/failed, so we do a polling on it. This is to avoid calling the hooks repeatedly
  • index — The index of the job
  • jid — The jid of the job in scheduler system
  • jid (int | str | none) — Get the jid of the job in scheduler system</>
  • jid_file (Path) — The jid file of the job</>
  • metadir — The metadir of the job
  • rc (int) — The return code of the job</>
  • rc_file (Path) — The rc file of the job</>
  • retry_dir (Path) — The retry directory of the job</>
  • script_file — Get the path to script file</>
  • signature_file — Get the path to the signature file</>
  • status (int) — Query the status of the job
    If the job is submitted, try to query it from the status file Make sure the status is updated by trap in wrapped script </>
  • status_file (Path) — The status file of the job</>
  • stderr_file (Path) — The stderr file of the job</>
  • stdout_file (Path) — The stdout file of the job</>
  • strcmd (str) — Get the string representation of the command</>
  • trial_count — The count for re-tries
Classes
  • ABCMeta — Metaclass for defining Abstract Base Classes (ABCs).</>
Methods
  • __repr__() (str) — repr of the job</>
  • cache() — write signature to signature file</>
  • clean(retry) — Clean up the meta files</>
  • log(level, msg, *args, limit, limit_indicator, logger) — Log message for the jobs</>
  • prepare(proc) — Prepare the job by given process</>
  • shebang(scheduler) (str) — The shebang of the wrapped script</>
  • wrapped_script(scheduler) (PathLike) — Get the wrapped script</>
method

cache()

</>

write signature to signature file

class

abc.ABCMeta(name, bases, namespace, **kwargs)

</>

Metaclass for defining Abstract Base Classes (ABCs).

Use this metaclass to create an ABC. An ABC can be subclassed directly, and then acts as a mix-in class. You can also register unrelated concrete classes (even built-in classes) and unrelated ABCs as 'virtual subclasses' -- these and their descendants will be considered subclasses of the registering ABC by the built-in issubclass() function, but the registering ABC won't show up in their MRO (Method Resolution Order) nor will method implementations defined by the registering ABC be callable (not even via super()).

Methods
staticmethod
register(cls, subclass)
</>

Register a virtual subclass of an ABC.

Returns the subclass, to allow usage as a class decorator.

staticmethod
__instancecheck__(cls, instance)
</>

Override for isinstance(instance, cls).

staticmethod
__subclasscheck__(cls, subclass)
</>

Override for issubclass(subclass, cls).

method

__repr__() → str

</>

repr of the job

method

shebang(scheduler) → str

</>

The shebang of the wrapped script

method

clean(retry=False)

</>

Clean up the meta files

Parameters
  • retry (optional) — Whether clean it for retrying
method

wrapped_script(scheduler)

</>

Get the wrapped script

Parameters
  • scheduler (Scheduler) — The scheduler
Returns (PathLike)

The path of the wrapped script

method

log(level, msg, *args, limit=3, limit_indicator=True, logger=<LoggerAdapter pipen.core (WARNING)>)

</>

Log message for the jobs

Parameters
  • level (int | str) — The log level of the record
  • msg (str) — The message to log
  • *args — The arguments to format the message
  • limit (int, optional) — limitation of the log (don't log for all jobs)
  • limit_indicator (bool, optional) — Whether to show an indicator saying the loghas been limited (the level of the indicator will be DEBUG)
  • logger (LoggerAdapter, optional) — The logger used to log
method

prepare(proc)

</>

Prepare the job by given process

Primarily prepare the script, and provide cmd to the job for xqute to wrap and run

Parameters
  • proc (Proc) — the process object