com.neurosys.etl.service

Class ETLRunner<IN,T>

java.lang.Object

com.neurosys.etl.service.ETLRunner<IN,T>

Type Parameters:

IN - as input type.

T - as the resultant output type for each tuple in the process.

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.context.ApplicationContextAware

public class ETLRunner<IN,T>
extends Object
implements org.springframework.context.ApplicationContextAware

Stand alone can trigger an ETL process without need for any UI.

Author:

Arjun Dhar

Constructor Summary

Constructors

Constructor and Description
ETLRunner()

Method Summary

Modifier and Type	Method and Description
protected void	handleCollectedExceptions(Collection<Throwable> errs) ETLRunnerOptions.isFailFast()=false then the exceptions per tuple collect, at the end of the process one can hook into them here.
protected boolean	handleTupleException(Exception e) How to handle an Exception @ a tuple level (only if ETLRunnerOptions.isFailFast() or critical exceptions.
protected void	init(AbstractReader<IN,T,Serializable> reader, ETLMeta meta) All Thread/Reading init based activities
protected void	injectContext(AbstractReader<IN,T,Serializable> reader) Inject context data into ParamsAware and ReaderAware like transformer implementation.
protected void	onEntity(T entity) Callback on creation/conversion of Entity from Tuple
protected void	onMetaProcessor(Object output) Callback on post processing of the entity; output of ETLMeta.getEntityProcessor()
protected void	processGeneratedEntity(T entity, AbstractReader<IN,T,Serializable> reader, ETLMeta meta, ETLRunnerOptions options) Process the final Entity produced from the ETL Process. This is a convenient way to simply provide Transformer based processing over input/reader produced output as AbstractReader are not required at times.
protected void	progressCallback(int percent, T lastEntityProcessed)
void	run(AbstractReader<IN,T,Serializable> reader, ETLMeta meta, ETLRunnerOptions options)
void	setApplicationContext(org.springframework.context.ApplicationContext context)
protected void	setupJob(AbstractReader<IN,T,Serializable> job, ETLMeta meta, ETLRunnerOptions options)
protected void	threadInit(AbstractReader<IN,T,Serializable> reader, ETLMeta meta) Init before the thread runs.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ETLRunner

public ETLRunner()

Method Detail

setupJob

protected void setupJob(AbstractReader<IN,T,Serializable> job,
                        ETLMeta meta,
                        ETLRunnerOptions options)
                 throws Exception

Throws:

Exception

init

protected void init(AbstractReader<IN,T,Serializable> reader,
                    ETLMeta meta)
             throws Exception

All Thread/Reading init based activities

Throws:

Exception

injectContext

protected void injectContext(AbstractReader<IN,T,Serializable> reader)

Inject context data into ParamsAware and ReaderAware like transformer implementation.

Parameters:

t - as Transformer

threadInit

protected void threadInit(AbstractReader<IN,T,Serializable> reader,
                          ETLMeta meta)

Init before the thread runs.

processGeneratedEntity

protected void processGeneratedEntity(T entity,
                                      AbstractReader<IN,T,Serializable> reader,
                                      ETLMeta meta,
                                      ETLRunnerOptions options)

Process the final Entity produced from the ETL Process.
This is a convenient way to simply provide Transformer based processing over input/reader produced output as AbstractReader are not required at times.

Parameters:

entity - as T @nullable true

handleTupleException

protected boolean handleTupleException(Exception e)

How to handle an Exception @ a tuple level (only if ETLRunnerOptions.isFailFast() or critical exceptions.

Parameters:

e - as Exception

Returns:

true if we wish to break the calling loop of tuples else false

Default value:

false

progressCallback

protected void progressCallback(int percent,
                                T lastEntityProcessed)

onEntity

protected void onEntity(T entity)

Callback on creation/conversion of Entity from Tuple

Parameters:

entity - as Entity type . If entity is null and ETLRunnerOptions.isFailFast()=false then there is an Exception.

See Also:

handleTupleException(Exception)

handleCollectedExceptions

protected void handleCollectedExceptions(Collection<Throwable> errs)

ETLRunnerOptions.isFailFast()=false then the exceptions per tuple collect, at the end of the process one can hook into them here.

Parameters:

errs - as Collection of Throwable

onMetaProcessor

protected void onMetaProcessor(Object output)

Callback on post processing of the entity; output of ETLMeta.getEntityProcessor()

Parameters:

output - as output Object

run

public void run(AbstractReader<IN,T,Serializable> reader,
                ETLMeta meta,
                ETLRunnerOptions options)
         throws Exception

Throws:

Exception

setApplicationContext

public void setApplicationContext(org.springframework.context.ApplicationContext context)
                           throws org.springframework.beans.BeansException

Specified by:

setApplicationContext in interface org.springframework.context.ApplicationContextAware

Throws:

org.springframework.beans.BeansException