xqute.xqute

module

xqute.xqute

</>

The xqute module

Classes

• Xqute — The main class of the package</>

class

xqute.xqute.Xqute(scheduler='local', plugins=None, workdir='./.xqute', submission_batch=8, error_strategy='ignore', num_retries=3, forks=1, scheduler_opts=None, jobname_prefix=None)

</>

The main class of the package

Attributes

• EMPTY_BUFFER_SLEEP_TIME — The time to sleep while waiting whenthe buffer is empty to wait for the jobs to be pushed
• _cancelling — A mark to mark whether a shutting down eventis triggered (True for natural cancelling, the signal for cancelling with a signal, SIGINT for example)
• buffer_queue — A buffer queue to save the pushed jobs
• jobs — The jobs registry
• name — The name, used in logger
• plugins — The plugins to be enabled or disabledto disable a plugin, using -plugin_name either all plugin names should be prefixed with '+'/'-' or none of them should
• queue — The job queue
• scheduler — The scheduler
• task — The task of producer and consumers

Parameters

• scheduler (str | Type[Scheduler], optional) — The scheduler class or name
• plugins (Optional, optional) — The plugins to be enabled or disabledto disable a plugin, using -plugin_name either all plugin names should be prefixed with '+'/'-' or none of them should
• workdir (str | PathType, optional) — The job meta directory
• submission_batch (int, optional) — The number of consumers to submit jobs. So that thesubmission process won't exhaust the local resources
• error_strategy (str, optional) — The strategy when there is error happened
• num_retries (int, optional) — Max number of retries when error_strategy is retry
• forks (int, optional) — Max number of job forks for scheduler
• scheduler_opts (Optional, optional) — Additional keyword arguments for scheduler

Methods

• cancel(sig) — Cancel the producer-consumer task</>
• put(cmd, envs) — Put a command into the buffer</>
• run_until_complete() — Wait until all jobs complete</>

method

cancel(sig=None)

</>

Cancel the producer-consumer task

self._cancelling will be set to signaled if sig is provided, otherwise it will be set to True

Parameters

• sig (signal.signals | none, optional) — Whether this cancelling is caused by a signal

method

put(cmd, envs=None)

</>

Put a command into the buffer

Parameters

• cmd (CommandType | Job) — The command
• envs (dict, optional) — The environment variables for the job

method

run_until_complete()

</>

Wait until all jobs complete