com.neurosys.etl.processors.batch

Class DaemonBufferSplitter<T>

java.lang.Object

com.neurosys.etl.process.EventAwareProcess

com.neurosys.etl.processors.batch.DaemonBufferSplitter<T>

Type Parameters:

T -

All Implemented Interfaces:

Process, BufferSplitter<T>, Serializable, Runnable

public abstract class DaemonBufferSplitter<T>
extends EventAwareProcess
implements BufferSplitter<T>, Runnable

A time aware reader (intended to be used a background low priority daemon service) that chunks out batches based on a Buffer Size & limit.
The Buffer is cleared/sent each time getBUFFER_HOLD_TIME() is reached or getMAX_BUFFER_SIZE() is reached
[Variable Name] ([Corresponding PortletConfig Value])

• MAX_BUFFER_SIZE (buffer.size.max)
• BUFFER_HOLD_TIME (buffer.time.max)

Author:

Arjun Dhar

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface com.neurosys.etl.process.Process

Process.Event, Process.EventPoint

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	alive
protected int	BUFFER_HOLD_TIME
protected boolean	BUFFERED_MODE
protected int	MAX_BUFFER_SIZE
protected int	MIN_BUFFER_SIZE

Fields inherited from class com.neurosys.etl.process.EventAwareProcess

listeners, processState

Constructor Summary

Constructors

Constructor and Description
DaemonBufferSplitter()
DaemonBufferSplitter(BlockingQueue<T> buffer)

Method Summary

Modifier and Type	Method and Description
int	getBUFFER_HOLD_TIME() If within BUFFER_HOLD_TIME if no alert is received then it will flush the buffer.
BlockingQueue<T>	getBuffer()
int	getMAX_BUFFER_SIZE() MAX_BUFFER_SIZE; when count exceeds this amount it splits a batch out buffer.size.max
void	kill()
abstract void	processBatch(List<T> batch) Callback on the batch created as a result of this process
void	refresh() Refresh the configs and setup the BufferSplitter thread again.
void	refreshConfigs() The thread code does not directly use the config, this may cause unexpected delays and problems depending on configs state.
void	run()
void	setBUFFER_HOLD_TIME(int bufferHoldTime) If within BUFFER_HOLD_TIME if no alert is received then it will flush the buffer.
void	setBuffer(BlockingQueue<T> buffer) Should be called by overriding class to set.
void	setMAX_BUFFER_SIZE(int mAXBUFFERSIZE) MAX_BUFFER_SIZE; when count exceeds this amount it splits a batch out buffer.size.max

Methods inherited from class com.neurosys.etl.process.EventAwareProcess

addListener, getListeners, onChangeHandler, setListeners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

alive

protected boolean alive

BUFFERED_MODE

protected boolean BUFFERED_MODE

MAX_BUFFER_SIZE

protected int MAX_BUFFER_SIZE

MIN_BUFFER_SIZE

protected int MIN_BUFFER_SIZE

BUFFER_HOLD_TIME

protected int BUFFER_HOLD_TIME

Constructor Detail

DaemonBufferSplitter

public DaemonBufferSplitter()

DaemonBufferSplitter

public DaemonBufferSplitter(BlockingQueue<T> buffer)

Method Detail

getBuffer

public BlockingQueue<T> getBuffer()

setBuffer

public void setBuffer(BlockingQueue<T> buffer)

Should be called by overriding class to set.
It can call and keep processing the batch created by the constraints of this Buffer Splitter

Parameters:

buffer -

processBatch

public abstract void processBatch(List<T> batch)

Callback on the batch created as a result of this process

Specified by:

processBatch in interface BufferSplitter<T>

Parameters:

batch - as List

kill

public void kill()
          throws Exception

Throws:

Exception

run

public void run()

Specified by:

run in interface Runnable

getMAX_BUFFER_SIZE

public final int getMAX_BUFFER_SIZE()

MAX_BUFFER_SIZE; when count exceeds this amount it splits a batch out
buffer.size.max

setMAX_BUFFER_SIZE

public final void setMAX_BUFFER_SIZE(int mAXBUFFERSIZE)

MAX_BUFFER_SIZE; when count exceeds this amount it splits a batch out
buffer.size.max

getBUFFER_HOLD_TIME

public final int getBUFFER_HOLD_TIME()

If within BUFFER_HOLD_TIME if no alert is received then it will flush the buffer.
buffer.time.max BUFFER_HOLD_TIME in TimeUnit.MILLISECONDS

setBUFFER_HOLD_TIME

public final void setBUFFER_HOLD_TIME(int bufferHoldTime)

If within BUFFER_HOLD_TIME if no alert is received then it will flush the buffer.
buffer.time.max BUFFER_HOLD_TIME in TimeUnit.MILLISECONDS

refreshConfigs

public void refreshConfigs()
                    throws Exception

The thread code does not directly use the config, this may cause unexpected delays and problems depending on configs state. Rather, we abstract the use of control/config variables in execution and refresh them when required.

Throws:

Exception

refresh

@PostConstruct
public void refresh()
                            throws Exception

Refresh the configs and setup the BufferSplitter thread again.
If instantiated by Spring, this method will auto kick in PostConstruct and start the Thread.

Throws:

Exception