Package nz.org.riskscape.engine.pipeline

Class RealizedPipeline

java.lang.Object

nz.org.riskscape.engine.pipeline.RealizedPipeline

All Implemented Interfaces:

AutoCloseable

public class RealizedPipeline
extends Object
implements AutoCloseable

A RealizedPipeline is the result of realizing (going from untyped to typed) a PipelineDeclaration against a project and a set of parameters via a PipelineRealizer. The structure of the pipeline is represented here as directed graph of RealizedSteps, where each step represents a typed transformation of data from one step to the next.

A RealizedPipeline can be given to a PipelineExecutor to execute the pipeline, writing results to a PipelineOutputContainer.

# Closing attached resources

Closing a RealizedPipeline will close all of its realized steps. One slightly awkward side effect of using an immutable-prototype pattern for building a realized pipeline is that it muddies the water for who is responsible for closing it.

My feeling is that as long as realized steps lazily allocate resources, that is, they only take them once execution starts, then this won't be a problem in practice: any orphaned steps won't need to be closed; only a pipeline and its steps that get executed will need to be closed.

If this starts to become a source of bugs, we could:

• Split this role up and have a RealizedPipelineBuilder and totally immutable RealizedPipeline. At this point only the RealizedPipeline is executable.

• Remove the AutoCloseable from RealizedPipeline and RealizedStep and consider changing the getResult() method to be more like a constructor, e.g. newRealized(ExecutionContext) and have that be the closable

Constructor Summary

Constructors

Constructor	Description
RealizedPipeline(ExecutionContext context, PipelineDeclaration pipeline, List<RealizedStep> steps)
RealizedPipeline(ExecutionContext context, PipelineDeclaration pipeline, List<RealizedStep> steps, List<Problem> problems)
RealizedPipeline(ExecutionContext context, PipelineDeclaration ast, List<RealizedStep> realizedSteps, List<Problem> problems, PipelineMetadata metadata)	Creates a new RealizedPipeline instance.

Method Summary

Modifier and Type	Method	Description
RealizedPipeline	add(RealizedStep newStep)
RealizedPipeline	addProblems(Problem... rest)
protected boolean	canEqual(Object other)
RealizedPipeline	clone()
void	close()
RealizedPipeline	drainWarnings(Consumer<Problem> problemConsumer)
static RealizedPipeline	empty(ExecutionContext context, PipelineDeclaration pipeline)
boolean	equals(Object o)
PipelineDeclaration	getAst()	The source code that was realized to produce this realized pipeline
ExecutionContext	getContext()
Set<RealizedStep>	getDependents(RealizedStep step)
Set<RealizedStep>	getEndSteps()
List<Problem>	getFailures()
PipelineMetadata	getMetadata()
List<RealizedStep>	getRealizedSteps()	The individually realized steps, in the order they were realized
Set<RealizedStep>	getStartSteps()
Optional<RealizedStep>	getStep(String stepName)
boolean	hasFailures()
int	hashCode()
RealizedPipeline	withMetadata(PipelineMetadata newMetadata)

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RealizedPipeline

public RealizedPipeline(ExecutionContext context,
 PipelineDeclaration pipeline,
 List<RealizedStep> steps)

RealizedPipeline

public RealizedPipeline(ExecutionContext context,
 PipelineDeclaration pipeline,
 List<RealizedStep> steps,
 List<Problem> problems)

RealizedPipeline

public RealizedPipeline(ExecutionContext context,
 PipelineDeclaration ast,
 List<RealizedStep> realizedSteps,
 List<Problem> problems,
 PipelineMetadata metadata)

Creates a new RealizedPipeline instance.

Parameters:

context -

ast - The source code that was realized to produce this realized pipeline

realizedSteps - The individually realized steps, in the order they were realized

problems - A set of problems associated with realizing the pipeline that weren't specific to a single step

metadata -

Method Details

empty

public static RealizedPipeline empty(ExecutionContext context,
 PipelineDeclaration pipeline)

Returns:

a pipeline with no steps.

close

public void close()

Specified by:

close in interface AutoCloseable

getStartSteps

public Set<RealizedStep> getStartSteps()

Returns:

the set of steps that have no dependencies

getEndSteps

public Set<RealizedStep> getEndSteps()

Returns:

the set of RealizedSteps that no other steps depend on

getDependents

public Set<RealizedStep> getDependents(RealizedStep step)

Returns:

a Set of RealizedSteps that have a dependency on the given step

getFailures

public List<Problem> getFailures()

Returns:

a list of Problems that directly caused this pipeline to fail

hasFailures

public boolean hasFailures()

Returns:

true if this realized pipeline has failed

getStep

public Optional<RealizedStep> getStep(String stepName)

Returns:

a step by its name within the pipeline (c.f. NamedStep), or empty if none can be found

clone

public RealizedPipeline clone()

Overrides:

clone in class Object

Returns:

a copy of this realized pipeline, copying references to any immutable members, and cloning anything that's mutable

drainWarnings

public RealizedPipeline drainWarnings(Consumer<Problem> problemConsumer)

Returns:

the RealizedPipeline with any warnings removed and passed to the problemConsumer

getMetadata

public PipelineMetadata getMetadata()

Returns:

metadata associated with this RealizedPipeline

withMetadata

public RealizedPipeline withMetadata(PipelineMetadata newMetadata)

Returns:

a copy of this pipeline, with different metadata to the original.

add

public RealizedPipeline add(RealizedStep newStep)

Returns:

a new RealizedPipeline that includes the extra step.

Throws:

IllegalArgumentException - if the step has dependencies not in this pipeline, or if it already contains a step with the same name.

addProblems

public RealizedPipeline addProblems(Problem... rest)

Returns:

a new pipeline that contains these extra problems

equals

public boolean equals(Object o)

Overrides:

equals in class Object

canEqual

protected boolean canEqual(Object other)

hashCode

public int hashCode()

Overrides:

hashCode in class Object

getContext

public ExecutionContext getContext()

getAst

public PipelineDeclaration getAst()

The source code that was realized to produce this realized pipeline

getRealizedSteps

public List<RealizedStep> getRealizedSteps()

The individually realized steps, in the order they were realized