this class is a base class for mapping data; see Mapper Examples for usage examples More...

Public Member Methods
	constructor (hash mapv, *hash opts)
	builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic More...

int	getCount ()
	returns the internal record count More...

string	getFieldName (string fname)
	returns a descriptive name of the given field if possible, otherwise returns the field name itself

*hash	getInputRecord ()
	returns the value of the `"input"` option

*hash	getOutputRecord ()
	returns the value of the `"output"` option

auto	getRuntime (string key)
	get current runtime option value for a key More...

	logOutput (hash h)
	calls the output logging closure or call reference (if any) to log the output record

list< hash >	mapAll (list recs)
	maps all input records and returns the mapped data as a list of output records More...

list< hash >	mapAll (hash recs)
	maps all input records and returns the mapped data as a list of output records More...

hash	mapData (hash rec)
	processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More...

hash	optionKeys ()
	returns a list of valid constructor options for this class (can be overridden in subclasses) More...

	replaceRuntime (*hash runtime)
	replaces runtime options More...

	resetCount ()
	resets the internal record count More...

	setRuntime (string key, auto value)
	set the runtime option with "key" to value "value" More...

	setRuntime (hash runtime)
	adds runtime options to the current runtime option hash More...

hash	validKeys ()
	returns a list of valid field keys for this class (can be overridden in subclasses) More...

hash	validTypes ()
	returns a list of valid field types for this class (can be overridden in subclasses) More...

Public Attributes
const	ConstantConflictList = ("name", "struct", "code", "default")
	field keys that conflict with "constant" and "index"

const	OptionKeys
	constructor option keys (can be extended by subclassing and reimplementing optionKeys())

const	OutputKeys
	output option keys

const	ValidKeys
	default known mapper hash field keys (can be extended by subclassing and reimplementing validKeys())

const	ValidTypes
	default known field types (can be extended by subclassing and reimplementing validTypes() and mapFieldType())

Private Member Methods
	checkInputField (string k, string name)
	raises an error if an invalid input field name is declared; only call this if "input" is defined

	checkMap ()
	verifies the input map in the constructor

	checkMapField (string k, reference< any > fh)
	perform per-field pre-processing on the passed map in the constructor More...

	checkTimezoneOption (hash opts, string rn)
	verifies a timezone constructor option

	constructor ()
	private constructor for subclasses

	convertToHash (int t, string k, reference< any > fh)
	convert a field definition to a hash if possible

	error (string fmt)
	throws a `MAP-ERROR` exception; prepends the map name to the description if known More...

	error2 (string ex, string fmt)
	throws the given exception; prepends the map name to the description if known

	fieldLengthError (string k, string val, int ix, int sze, int maxlen, hash rc)
	called when a field exceeds its maximum length when processing hashes of lists

hash	mapDataIntern (hash rec)
	processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More...

nothing	mapFieldIntern (reference< hash > h, string key, hash rec, bool do_list, int list_size)
	maps a single field to the target More...

	mapFieldType (string key, hash m, reference< any > v, hash rec)
	performs type handling

auto	mapSubclass (hash m, auto v)
	to be overridden as necessary in subclasses

	setup (hash mapv, *hash opts)
	sets up the mapper object before checking the mapper hash

string	truncateField (string k, string val, int ix, int sze, int maxlen)
	called to truncate fields when processing hashes of lists

Private Attributes
bool	allow_dot = False
	do not assume struct when field names have a `"."` in them; instead allow input field names to have a `"."` in them

bool	allow_output_dot = False
	do not assume structured/hash output when output field names have a `"."` in them; instead allow output field names to have a `"."` in them

hash	consth
	map of constant fields

int	count = 0
	count of records mapped

*string	date_format
	the global date format for parsing dates

string	encoding = "utf-8"
	the output character encoding; if not given then the output encoding is assumed to be UTF-8

hash	identh
	map of fields to be mapped 1:1 input -> output

*list	identl
	list of fields to be mapped 1:1 input -> output

*code	info_log
	an optional info logging callback; must accept a sprintf()-style format specifier and optional arguments

*hash	input
	an optional description of possible input hash keys

*code	input_log
	an optional input data logging callback; must accept a hash giving the input data hash

Qore::TimeZone	input_timezone = TimeZone::get()
	the timezone for input fields in case of parsing text values; if not set defaults to the current TimeZone (see Qore::TimeZone::get())

bool	m_empty_strings_to_nothing = False
	flag to enforce deletion of the empty string in the output record More...

*hash	m_runtime
	current runtime values More...

hash	mapc
	the hash providing output field names and mappings

hash	mapd
	the hash with a subset of the mappings used dynamically

hash	mapo
	the hash of output records for key order

*string	name
	the optional name for the object (for example a table name); will be prepended to field names in error messages

*string	number_format
	the global number format for parsing numbers

*hash	output
	an optional description of the output data structure

*code	output_log
	an optional output data logging callback; must accept a hash giving the output data hash

hash	rconsth
	map of constant runtime fields

*Qore::TimeZone	timezone
	an optional timezone for output date fields

bool	trunc_all = False
	truncate all option

Detailed Description

this class is a base class for mapping data; see Mapper Examples for usage examples

Member Function Documentation

◆ checkMapField()

Mapper::Mapper::checkMapField	(	string	k,
		reference< any >	fh
	)

private

perform per-field pre-processing on the passed map in the constructor

Parameters

k	the field name
fh	a reference to the field's value in the map; will be converted to a hash

◆ constructor()

Mapper::Mapper::constructor	(	hash	mapv,
		*hash	opts
	)

builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic

Example:: const DataMap = (
# output field: "id" mapper from the "Id" element of any "^attributes^" hash in the input record
"id": "^attributes^.Id",
# output field: "name": maps from an input field with the same name (no translations are made)
"name": True,
# output field: "explicit_count": maps from the input "Count" field, if any value is present then it is converted to an integer
"explicit_count": ("type": "int", "name": "Count"),
# output field: "implicit_count": runs the given code on the input record and retuns the result, the code returns the number of "Products" sub-records
"implicit_count": int sub (any ignored, hash rec) { return rec.Products.size(); },
# output field: "order_date": converts the "OrderDate" string input field to a date in the specified format
"order_date": ("name": "OrderDate", "date_format": "DD.MM.YYYY HH:mm:SS.us"),
);
Mapper mapv(DataMap);

Parameters

mapv	a hash providing field mappings; each hash key is the name of the output field; each value is either True (meaning no translations are done; the data is copied 1:1) or a hash describing the mapping; see Mapper Specification Format for detailed documentation for this option
opts	an optional hash of options for the mapper; see Mapper Options for a description of valid mapper options

Exceptions

MAP-ERROR the map hash has a logical error (ex: "trunc" key given without "maxlen", invalid map key)

◆ error()

Mapper::Mapper::error ( string fmt )

private

throws a MAP-ERROR exception; prepends the map name to the description if known

if this method is subclassed, it must also cause an exception to be thrown

◆ getCount()

int Mapper::Mapper::getCount ( )

returns the internal record count

See also: resetCount()

◆ getRuntime()

auto Mapper::Mapper::getRuntime ( string key )

get current runtime option value for a key

Parameters

key	the runtime option key

Returns: a runtime value if the key exists in the current runtime option hash and is set

See also

Since: Mapper 1.1

◆ mapAll() [1/2]

list<hash> Mapper::Mapper::mapAll ( list recs )

maps all input records and returns the mapped data as a list of output records

this method applies the mapData() method to all input records and returns the resulting list

Parameters

recs	the list of input records

Returns: the mapped data as a list of output records

Exceptions

MISSING-INPUT	a field marked mandatory is missing
STRING-TOO-LONG	a field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBER	the field is marked as numeric but the input value contains non-numeric data

◆ mapAll() [2/2]

list<hash> Mapper::Mapper::mapAll ( hash recs )

maps all input records and returns the mapped data as a list of output records

this method applies the mapData() method to all input records and returns the resulting list

Parameters

recs	a hash of lists of input records

Returns: the mapped data as a list of output records

Exceptions

MISSING-INPUT	a field marked mandatory is missing
STRING-TOO-LONG	a field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBER	the field is marked as numeric but the input value contains non-numeric data

◆ mapData()

hash Mapper::Mapper::mapData ( hash rec )

processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.

Parameters

rec	the record to translate

Returns: a hash of field values in the target format based on the input data and processed according to the logic in the map hash

Exceptions

MISSING-INPUT	a field marked mandatory is missing
STRING-TOO-LONG	a field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBER	the field is marked as numeric but the input value contains non-numeric data

Note

each time this method is executed successfully, the record count is updated (see getCount() and resetCount())
uses mapDataIntern() to map the data, then logOutput() is called for each output row

◆ mapDataIntern()

hash Mapper::Mapper::mapDataIntern ( hash rec )

private

processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.

Parameters

rec	the record to translate

Returns: a hash of field values in the target format based on the input data and processed according to the logic in the map hash

Exceptions

MISSING-INPUT	a field marked mandatory is missing
STRING-TOO-LONG	a field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBER	the field is marked as numeric but the input value contains non-numeric data

Note

each time this method is executed successfully, the record count is updated (see getCount() and resetCount())
this is the same as mapData() except no output logging is performed

◆ mapFieldIntern()

nothing Mapper::Mapper::mapFieldIntern	(	reference< hash >	h,
		string	key,
		hash	rec,
		bool	do_list,
		int	list_size
	)

private

maps a single field to the target

Performs the actual mapping

Parameters

h	the hash to be updated with the mapped key/value pair; Depending on the mapper specification and the input, it can contain a bulk record (hash of lists) or a hash of both list and non-list keys (these are considered a "constants" in the output record.
key	the column name (hash key) to be mapped (target field)
rec	input record - either single record of hash of lists (batch); to increase performance, the input type (single record vs. batch) is determined by the do_list parameter. In case of bulk input, all the lists are supposed to have the same length list_size
do_list	- whether the input record rec is single record or bulk format (hash of lists)
list_size	- size of the lists in case the input is in bulk format (all lists must have the same length, asserted inside the method).

Note: it is assumed that list_size > 0 whenever do_list is Qore::True; if do_list == Qore::True, the 'rec' can contain a mix of lists and non-list values - the non-list values are used as constants in the mapping - as if expanded to lists with all values identical.

◆ optionKeys()

hash Mapper::Mapper::optionKeys ( )

returns a list of valid constructor options for this class (can be overridden in subclasses)

Returns: a list of valid constructor options for this class (can be overridden in subclasses)

◆ replaceRuntime()

Mapper::Mapper::replaceRuntime ( *hash runtime )

replaces runtime options

Parameters

runtime a hash of runtime options to use to replace the current runtime option hash

Note: this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data

See also

Since: Mapper 1.1

Deprecated:: use Mapper construction options to set the runtime options.

Exceptions

RUNTIME-OPTION-CHANGED the option has changed during while already processing the data.

◆ resetCount()

Mapper::Mapper::resetCount ( )

resets the internal record count

See also: getCount()

◆ setRuntime() [1/2]

Mapper::Mapper::setRuntime	(	string	key,
		auto	value
	)

set the runtime option with "key" to value "value"

Parameters

key	a string with valid runtime key
value	anything passed to the current runtime `key`

Note: this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data

See also

Since: Mapper 1.1

Deprecated:: use Mapper construction options to set the runtime options.

Exceptions

RUNTIME-OPTION-CHANGED the option has changed during while already processing the data.

◆ setRuntime() [2/2]

Mapper::Mapper::setRuntime ( hash runtime )

adds runtime options to the current runtime option hash

Parameters

runtime a hash of runtime options to add to the current runtime option hash

Note: this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data

See also

Since: Mapper 1.1

Deprecated:: use Mapper construction options to set the runtime options.

Exceptions

RUNTIME-OPTION-CHANGED the option has changed during while already processing the data.

◆ validKeys()

hash Mapper::Mapper::validKeys ( )

returns a list of valid field keys for this class (can be overridden in subclasses)

Returns: a list of valid field keys for this class (can be overridden in subclasses)

◆ validTypes()

hash Mapper::Mapper::validTypes ( )

returns a list of valid field types for this class (can be overridden in subclasses)

Returns: a list of valid types for this class (can be overridden in subclasses)

Member Data Documentation

◆ m_empty_strings_to_nothing

bool Mapper::Mapper::m_empty_strings_to_nothing = False

private

flag to enforce deletion of the empty string in the output record

Since: Mapper 1.1

◆ m_runtime

*hash Mapper::Mapper::m_runtime

private

current runtime values

Since: Mapper 1.1

Public Member Methods

Public Attributes

Private Member Methods

Private Attributes

Detailed Description

Member Function Documentation

◆ checkMapField()

◆ constructor()

◆ error()

◆ getCount()

◆ getRuntime()

◆ mapAll() [1/2]

◆ mapAll() [2/2]

◆ mapData()

◆ mapDataIntern()

◆ mapFieldIntern()

◆ optionKeys()

◆ replaceRuntime()

◆ resetCount()

◆ setRuntime() [1/2]

◆ setRuntime() [2/2]

◆ validKeys()

◆ validTypes()

Member Data Documentation

◆ m_empty_strings_to_nothing

◆ m_runtime