DataProvider::AbstractDataProcessor Class Reference

Defines an abstract class for accepting data and outputting optionally transformed or filtered data. More...

Public Member Methods
	flush (code enqueue)
	This method is called for each pipeline element to permit flushing of any pending data at the end of processing. More...
AbstractDataProviderType	getExpectedType ()
	Returns the expected type of data to be submitted, if available. More...
AbstractDataProviderType	getReturnType ()
	Returns the type of data that will be returned, if available. More...
	setThreadLocalData (*hash< auto > thread_local_data)
	Sets thread-local data to set before running the processor.
	submit (code enqueue, auto _data)
	Submits the data for processing. More...
bool	supportsBulkApi ()
	Returns True if the data processor supports bulk operation. More...

Private Member Methods
	flushImpl (code enqueue)
	This method is called for each pipeline element to permit flushing of any pending data at the end of processing. More...
*AbstractDataProviderType	getExpectedTypeImpl ()
	Returns the expected type of data to be submitted, if available. More...
*AbstractDataProviderType	getReturnTypeImpl ()
	Returns the type of data that will be returned, if available. More...
abstract	submitImpl (code enqueue, auto _data)
	Submits the data for processing. More...
abstract bool	supportsBulkApiImpl ()
	Returns True if the data processor supports bulk operation. More...

Private Attributes
*hash< auto >	thread_local_data
	Thread-local data to set before running the processor.
*list< string >	thread_local_data_keys
	Thread-local data keys from thread_local_data.

Detailed Description

Defines an abstract class for accepting data and outputting optionally transformed or filtered data.

Member Function Documentation

flush()

DataProvider::AbstractDataProcessor::flush

(

code

enqueue

)

This method is called for each pipeline element to permit flushing of any pending data at the end of processing.

Parameters

enqueue

a closure taking a single arugment that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated, then call it once for each generated record; prototype: code enqueue = sub (auto qdata) {}

Note

• Calls flushImpl() on the data to do the actual processing
• Accept and return type information is not enforced in this method; it must be enforced in flushImpl()

flushImpl()

DataProvider::AbstractDataProcessor::flushImpl ( code  enqueue )

private

This method is called for each pipeline element to permit flushing of any pending data at the end of processing.

Parameters

enqueue

a closure taking a single arugment that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated, then call it once for each generated record; prototype: code enqueue = sub (auto qdata) {}

Note

• The default implementation in the base class does nothing; to submit pending data for processing by subsequent elements in the pipeline when pipeline processing is finished, call enqueue with the pending data
• No data should be cached or pending after this call; if the object requires closing or purging of internal state to make it reusable, it should be performed in this method

getExpectedType()

AbstractDataProviderType DataProvider::AbstractDataProcessor::getExpectedType

(

)

Returns the expected type of data to be submitted, if available.

Returns

the expected type of data to be submitted, if available

Note

Calls getExpectedTypeImpl() to provide the return value

getExpectedTypeImpl()

*AbstractDataProviderType DataProvider::AbstractDataProcessor::getExpectedTypeImpl ( )

private

Returns the expected type of data to be submitted, if available.

This base class method returns NOTHING; reimplement in subclasses to provide a type

getReturnType()

AbstractDataProviderType DataProvider::AbstractDataProcessor::getReturnType

(

)

Returns the type of data that will be returned, if available.

Returns

the type of data that will be returned, if available

Note

Calls getReturnTypeImpl() to provide the return value

getReturnTypeImpl()

*AbstractDataProviderType DataProvider::AbstractDataProcessor::getReturnTypeImpl ( )

private

Returns the type of data that will be returned, if available.

This base class method returns NOTHING; reimplement in subclasses to provide a type

submit()

DataProvider::AbstractDataProcessor::submit	(	code	enqueue,
		auto	_data
	)

Submits the data for processing.

Parameters

enqueue	a closure taking a single arugment that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated from the input data, then call it once for each generated record; prototype: code enqueue = sub (auto qdata) {}
_data	the data to process

Sets thread-local data before running and clears on exit if any is set

Note

• Calls submitImpl() on the data to do the actual processing
• Accept and return type information is not enforced in this method; it must be enforced in submitImpl()

submitImpl()

abstract DataProvider::AbstractDataProcessor::submitImpl ( code  enqueue, auto  _data  )

privatepure virtual

Submits the data for processing.

Parameters

enqueue	a closure taking a single arugment that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated from the input data, then call it once for each generated record; prototype: code enqueue = sub (auto qdata) {}
_data	the data to process

supportsBulkApi()

bool DataProvider::AbstractDataProcessor::supportsBulkApi

(

)

Returns True if the data processor supports bulk operation.

Returns

True if the data processor supports bulk operation

Note

Calls supportsBulkApiImpl() to return the answer

supportsBulkApiImpl()

abstract bool DataProvider::AbstractDataProcessor::supportsBulkApiImpl ( )

privatepure virtual

Returns True if the data processor supports bulk operation.

Returns

True if the data processor supports bulk operation

See also

dataprovider_pipeline_bulk_processing