pipen.procgroup
Process group that contains a set of processes.
It can be easily used to create a pipeline that runs independently or integrated into a larger pipeline.
Runs directly:
>>> proc_group = ProcGroup(<options>)
>>> proc_group.as_pipen(<pipeline options>).set_data(<data>).run()
Integrated into a larger pipeline
>>> proc_group = ProcGroup(<options>)
>>> # proc could be a process within the larger pipeline
>>> proc.requires = prog_group.<proc>
To add a process to the proc group, use the add_proc
method:
>>> class MyProcGroup(ProcGroup):
>>> ...
>>>
>>> proc_group = MyProcGroup(...)
>>> @proc_group.add_proc
>>> class MyProc(Proc):
>>> ...
Or add a process at runtime:
>>> class MyProcGroup(ProcGroup):
>>> ...
>>>
>>> @ProcGroup.add_proc
>>> def my_proc(self):
>>> class MyProc(Proc):
>>> # You may use self.options here
>>> ...
>>> return MyProc
>>> proc_group = MyProcGroup(...)
ProcGropuMeta
— Meta class for ProcGroup</>ProcGroup
— A group of processes that can be run independently orintegrated into a larger pipeline. </>
pipen.procgroup.
ProcGropuMeta
(
name
, bases
, namespace
, **kwargs
)
Meta class for ProcGroup
__call__
(
cls
,*args
,**kwds
)
— Make sure Proc subclasses are singletons</>__instancecheck__
(
cls
,instance
)
— Override for isinstance(instance, cls).</>__subclasscheck__
(
cls
,subclass
)
— Override for issubclass(subclass, cls).</>register
(
cls
,subclass
)
— Register a virtual subclass of an ABC.</>
register
(
cls
, subclass
)
Register a virtual subclass of an ABC.
Returns the subclass, to allow usage as a class decorator.
__instancecheck__
(
cls
, instance
)
Override for isinstance(instance, cls).
__subclasscheck__
(
cls
, subclass
)
Override for issubclass(subclass, cls).
pipen.procgroup.
ProcGroup
(
*args
, **kwds
)
A group of processes that can be run independently orintegrated into a larger pipeline.
ProcGropuMeta
— Meta class for ProcGroup</>
pipen.procgroup.
ProcGropuMeta
(
name
, bases
, namespace
, **kwargs
)
Meta class for ProcGroup
__call__
(
cls
,*args
,**kwds
)
— Make sure Proc subclasses are singletons</>__instancecheck__
(
cls
,instance
)
— Override for isinstance(instance, cls).</>__subclasscheck__
(
cls
,subclass
)
— Override for issubclass(subclass, cls).</>register
(
cls
,subclass
)
— Register a virtual subclass of an ABC.</>
register
(
cls
, subclass
)
Register a virtual subclass of an ABC.
Returns the subclass, to allow usage as a class decorator.
__instancecheck__
(
cls
, instance
)
Override for isinstance(instance, cls).
__subclasscheck__
(
cls
, subclass
)
Override for issubclass(subclass, cls).
__init_subclass__
(
)
This method is called when a class is subclassed.
The default implementation does nothing. It may be overridden to extend subclasses.
Add a process to the proc group
It works either as a decorator to the process directly or as a decorator to a method that returns the process.
self_or_method
(Union) — The proc group instance or a method thatreturns the processproc
(Optional, optional) — The process class ifself_or_method
is the proc group
The process class if self_or_method
is the proc group, ora cached property that returns the process class
Convert the pipeline to a Pipen instance
name
(str | none, optional) — The name of the pipelinedesc
(str | none, optional) — The description of the pipelineoutdir
(str | os.pathlike | none, optional) — The output directory of the pipeline**kwargs
— The keyword arguments to pass to Pipen
The Pipen instance