Skip to content

pipen.procgroup

module

pipen.procgroup

</>

Process group that contains a set of processes.

It can be easily used to create a pipeline that runs independently or integrated into a larger pipeline.

Runs directly: 

>>> proc_group = ProcGroup(<options>)
>>> proc_group.as_pipen(<pipeline options>).set_data(<data>).run()

Integrated into a larger pipeline 

>>> proc_group = ProcGroup(<options>)
>>> # proc could be a process within the larger pipeline
>>> proc.requires = prog_group.<proc>

To add a process to the proc group, use the add_proc method: 

>>> class MyProcGroup(ProcGroup):
>>>     ...
>>>
>>> proc_group = MyProcGroup(...)
>>> @proc_group.add_proc
>>> class MyProc(Proc):
>>>     ...

Or add a process at runtime: 

>>> class MyProcGroup(ProcGroup):
>>>     ...
>>>
>>>     @ProcGroup.add_proc
>>>     def my_proc(self):
>>>         class MyProc(Proc):
>>>             # You may use self.options here
>>>             ...
>>>         return MyProc
>>> proc_group = MyProcGroup(...)

Classes
  • ProcGropuMeta — Meta class for ProcGroup</>
  • ProcGroup — A group of processes that can be run independently orintegrated into a larger pipeline. </>
class

pipen.procgroup.ProcGropuMeta(name, bases, namespace, **kwargs)

</>
Bases
abc.ABCMeta

Meta class for ProcGroup

Methods
  • __call__(cls, *args, **kwds) — Make sure Proc subclasses are singletons</>
staticmethod

__call__(cls, *args, **kwds)

</>

Make sure Proc subclasses are singletons

Parameters
  • *args — and
  • **kwds — Arguments for the constructor
Returns

The Proc instance

class

pipen.procgroup.ProcGroup(*args, **kwds)

</>

A group of processes that can be run independently orintegrated into a larger pipeline.

Classes
Methods
  • __init_subclass__() — This method is called when a class is subclassed.</>
  • add_proc(self_or_method, proc) (Union) — Add a process to the proc group</>
  • as_pipen(name, desc, outdir, **kwargs) (Pipen) — Convert the pipeline to a Pipen instance</>
class

pipen.procgroup.ProcGropuMeta(name, bases, namespace, **kwargs)

</>
Bases
abc.ABCMeta

Meta class for ProcGroup

Methods
  • __call__(cls, *args, **kwds) — Make sure Proc subclasses are singletons</>
staticmethod
__call__(cls, *args, **kwds)
</>

Make sure Proc subclasses are singletons

Parameters
  • *args — and
  • **kwds — Arguments for the constructor
Returns

The Proc instance

classmethod

__init_subclass__()

</>

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

staticmethod

add_proc(self_or_method, proc=None)

</>

Add a process to the proc group

It works either as a decorator to the process directly or as a decorator to a method that returns the process.

Parameters
  • self_or_method (Union) — The proc group instance or a method thatreturns the process
  • proc (Optional, optional) — The process class if self_or_method is the proc group
Returns (Union)

The process class if self_or_method is the proc group, ora cached property that returns the process class

method

as_pipen(name=None, desc=None, outdir=None, **kwargs)

</>

Convert the pipeline to a Pipen instance

Parameters
  • name (str | none, optional) — The name of the pipeline
  • desc (str | none, optional) — The description of the pipeline
  • outdir (str | os.pathlike | none, optional) — The output directory of the pipeline
  • **kwargs — The keyword arguments to pass to Pipen
Returns (Pipen)

The Pipen instance