Qore Mapper Module Reference  1.4
Mapper::Mapper Class Reference

this class is a base class for mapping data; see Mapper Examples for usage examples More...

Public Member Functions

 constructor (hash mapv, *hash opts)
 builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic More...
 
int getCount ()
 returns the internal record count More...
 
string getFieldName (string fname)
 returns a descriptive name of the given field if possible, otherwise returns the field name itself
 
*hash getInputRecord ()
 returns the value of the "input" option
 
*hash getOutputRecord ()
 returns the value of the "output" option
 
auto getRuntime (string key)
 get current runtime option value for a key More...
 
 logOutput (hash h)
 calls the output logging closure or call reference (if any) to log the output record
 
list< hashmapAll (list recs)
 maps all input records and returns the mapped data as a list of output records More...
 
list< hashmapAll (hash recs)
 maps all input records and returns the mapped data as a list of output records More...
 
hash mapData (hash rec)
 processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More...
 
hash optionKeys ()
 returns a list of valid constructor options for this class (can be overridden in subclasses) More...
 
 replaceRuntime (*hash runtime)
 replaces runtime options More...
 
 resetCount ()
 resets the internal record count More...
 
 setRuntime (string key, auto value)
 set the runtime option with "key" to value "value" More...
 
 setRuntime (hash runtime)
 adds runtime options to the current runtime option hash More...
 
hash validKeys ()
 returns a list of valid field keys for this class (can be overridden in subclasses) More...
 
hash validTypes ()
 returns a list of valid field types for this class (can be overridden in subclasses) More...
 

Public Attributes

const ConstantConflictList = ("name", "struct", "code", "default")
 field keys that conflict with "constant" and "index"
 
const OptionKeys
 constructor option keys (can be extended by subclassing and reimplementing optionKeys())
 
const OutputKeys
 output option keys
 
const ValidKeys
 default known mapper hash field keys (can be extended by subclassing and reimplementing validKeys())
 
const ValidTypes
 default known field types (can be extended by subclassing and reimplementing validTypes() and mapFieldType())
 

Private Member Functions

 checkInputField (string k, string name)
 raises an error if an invalid input field name is declared; only call this if "input" is defined
 
 checkMap ()
 verifies the input map in the constructor
 
 checkMapField (string k, reference< any > fh)
 perform per-field pre-processing on the passed map in the constructor More...
 
 checkTimezoneOption (hash opts, string rn)
 verifies a timezone constructor option
 
 constructor ()
 private constructor for subclasses
 
 convertToHash (int t, string k, reference< any > fh)
 convert a field definition to a hash if possible
 
 error (string fmt)
 throws a MAP-ERROR exception; prepends the map name to the description if known More...
 
 error2 (string ex, string fmt)
 throws the given exception; prepends the map name to the description if known
 
 fieldLengthError (string k, string val, int ix, int sze, int maxlen, hash rc)
 called when a field exceeds its maximum length when processing hashes of lists
 
hash mapDataIntern (hash rec)
 processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More...
 
nothing mapFieldIntern (reference< hash > h, string key, hash rec, bool do_list, int list_size)
 maps a single field to the target More...
 
 mapFieldType (string key, hash m, reference< any > v, hash rec)
 performs type handling
 
auto mapSubclass (hash m, auto v)
 to be overridden as necessary in subclasses
 
 setup (hash mapv, *hash opts)
 sets up the mapper object before checking the mapper hash
 
string truncateField (string k, string val, int ix, int sze, int maxlen)
 called to truncate fields when processing hashes of lists
 

Private Attributes

bool allow_dot = False
 do not assume struct when field names have a "." in them; instead allow input field names to have a "." in them
 
bool allow_output_dot = False
 do not assume structured/hash output when output field names have a "." in them; instead allow output field names to have a "." in them
 
hash consth
 map of constant fields
 
int count = 0
 count of records mapped
 
*string date_format
 the global date format for parsing dates
 
string encoding = "utf-8"
 the output character encoding; if not given then the output encoding is assumed to be UTF-8
 
hash identh
 map of fields to be mapped 1:1 input -> output
 
*list identl
 list of fields to be mapped 1:1 input -> output
 
*code info_log
 an optional info logging callback; must accept a sprintf()-style format specifier and optional arguments
 
*hash input
 an optional description of possible input hash keys
 
*code input_log
 an optional input data logging callback; must accept a hash giving the input data hash
 
Qore::TimeZone input_timezone = TimeZone::get()
 the timezone for input fields in case of parsing text values; if not set defaults to the current TimeZone (see Qore::TimeZone::get())
 
bool m_empty_strings_to_nothing = False
 flag to enforce deletion of the empty string in the output record More...
 
*hash m_runtime
 current runtime values More...
 
hash mapc
 the hash providing output field names and mappings
 
hash mapd
 the hash with a subset of the mappings used dynamically
 
hash mapo
 the hash of output records for key order
 
*string name
 the optional name for the object (for example a table name); will be prepended to field names in error messages
 
*string number_format
 the global number format for parsing numbers
 
*hash output
 an optional description of the output data structure
 
*code output_log
 an optional output data logging callback; must accept a hash giving the output data hash
 
hash rconsth
 map of constant runtime fields
 
*Qore::TimeZone timezone
 an optional timezone for output date fields
 
bool trunc_all = False
 truncate all option
 

Detailed Description

this class is a base class for mapping data; see Mapper Examples for usage examples

Member Function Documentation

◆ checkMapField()

Mapper::Mapper::checkMapField ( string  k,
reference< any >  fh 
)
private

perform per-field pre-processing on the passed map in the constructor

Parameters
kthe field name
fha reference to the field's value in the map; will be converted to a hash

◆ constructor()

Mapper::Mapper::constructor ( hash  mapv,
*hash  opts 
)

builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic

Example:
const DataMap = (
# output field: "id" mapper from the "Id" element of any "^attributes^" hash in the input record
"id": "^attributes^.Id",
# output field: "name": maps from an input field with the same name (no translations are made)
"name": True,
# output field: "explicit_count": maps from the input "Count" field, if any value is present then it is converted to an integer
"explicit_count": ("type": "int", "name": "Count"),
# output field: "implicit_count": runs the given code on the input record and retuns the result, the code returns the number of "Products" sub-records
"implicit_count": int sub (any ignored, hash rec) { return rec.Products.size(); },
# output field: "order_date": converts the "OrderDate" string input field to a date in the specified format
"order_date": ("name": "OrderDate", "date_format": "DD.MM.YYYY HH:mm:SS.us"),
);
Mapper mapv(DataMap);
Parameters
mapva hash providing field mappings; each hash key is the name of the output field; each value is either True (meaning no translations are done; the data is copied 1:1) or a hash describing the mapping; see Mapper Specification Format for detailed documentation for this option
optsan optional hash of options for the mapper; see Mapper Options for a description of valid mapper options
Exceptions
MAP-ERRORthe map hash has a logical error (ex: "trunc" key given without "maxlen", invalid map key)

◆ error()

Mapper::Mapper::error ( string  fmt)
private

throws a MAP-ERROR exception; prepends the map name to the description if known

if this method is subclassed, it must also cause an exception to be thrown

◆ getCount()

int Mapper::Mapper::getCount ( )

returns the internal record count

See also
resetCount()

◆ getRuntime()

auto Mapper::Mapper::getRuntime ( string  key)

get current runtime option value for a key

Parameters
keythe runtime option key
Returns
a runtime value if the key exists in the current runtime option hash and is set
See also
Since
Mapper 1.1

◆ mapAll() [1/2]

list<hash> Mapper::Mapper::mapAll ( list  recs)

maps all input records and returns the mapped data as a list of output records

this method applies the mapData() method to all input records and returns the resulting list

Parameters
recsthe list of input records
Returns
the mapped data as a list of output records
Exceptions
MISSING-INPUTa field marked mandatory is missing
STRING-TOO-LONGa field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBERthe field is marked as numeric but the input value contains non-numeric data

◆ mapAll() [2/2]

list<hash> Mapper::Mapper::mapAll ( hash  recs)

maps all input records and returns the mapped data as a list of output records

this method applies the mapData() method to all input records and returns the resulting list

Parameters
recsa hash of lists of input records
Returns
the mapped data as a list of output records
Exceptions
MISSING-INPUTa field marked mandatory is missing
STRING-TOO-LONGa field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBERthe field is marked as numeric but the input value contains non-numeric data

◆ mapData()

hash Mapper::Mapper::mapData ( hash  rec)

processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.

Parameters
recthe record to translate
Returns
a hash of field values in the target format based on the input data and processed according to the logic in the map hash
Exceptions
MISSING-INPUTa field marked mandatory is missing
STRING-TOO-LONGa field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBERthe field is marked as numeric but the input value contains non-numeric data
Note

◆ mapDataIntern()

hash Mapper::Mapper::mapDataIntern ( hash  rec)
private

processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.

Parameters
recthe record to translate
Returns
a hash of field values in the target format based on the input data and processed according to the logic in the map hash
Exceptions
MISSING-INPUTa field marked mandatory is missing
STRING-TOO-LONGa field value exceeds the maximum value and the 'trunc' key is not set
INVALID-NUMBERthe field is marked as numeric but the input value contains non-numeric data
Note
  • each time this method is executed successfully, the record count is updated (see getCount() and resetCount())
  • this is the same as mapData() except no output logging is performed

◆ mapFieldIntern()

nothing Mapper::Mapper::mapFieldIntern ( reference< hash h,
string  key,
hash  rec,
bool  do_list,
int  list_size 
)
private

maps a single field to the target

Performs the actual mapping

Parameters
hthe hash to be updated with the mapped key/value pair; Depending on the mapper specification and the input, it can contain a bulk record (hash of lists) or a hash of both list and non-list keys (these are considered a "constants" in the output record.
keythe column name (hash key) to be mapped (target field)
recinput record - either single record of hash of lists (batch); to increase performance, the input type (single record vs. batch) is determined by the do_list parameter. In case of bulk input, all the lists are supposed to have the same length list_size
do_list- whether the input record rec is single record or bulk format (hash of lists)
list_size- size of the lists in case the input is in bulk format (all lists must have the same length, asserted inside the method).
Note
it is assumed that list_size > 0 whenever do_list is Qore::True
if do_list == Qore::True, the 'rec' can contain a mix of lists and non-list values - the non-list values are used as constants in the mapping - as if expanded to lists with all values identical.

◆ optionKeys()

hash Mapper::Mapper::optionKeys ( )

returns a list of valid constructor options for this class (can be overridden in subclasses)

Returns
a list of valid constructor options for this class (can be overridden in subclasses)

◆ replaceRuntime()

Mapper::Mapper::replaceRuntime ( *hash  runtime)

replaces runtime options

Parameters
runtimea hash of runtime options to use to replace the current runtime option hash
Note
this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data
See also
Since
Mapper 1.1
Deprecated:
use Mapper construction options to set the runtime options.
Exceptions
RUNTIME-OPTION-CHANGEDthe option has changed during while already processing the data.

◆ resetCount()

Mapper::Mapper::resetCount ( )

resets the internal record count

See also
getCount()

◆ setRuntime() [1/2]

Mapper::Mapper::setRuntime ( string  key,
auto  value 
)

set the runtime option with "key" to value "value"

Parameters
keya string with valid runtime key
valueanything passed to the current runtime key
Note
this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data
See also
Since
Mapper 1.1
Deprecated:
use Mapper construction options to set the runtime options.
Exceptions
RUNTIME-OPTION-CHANGEDthe option has changed during while already processing the data.

◆ setRuntime() [2/2]

Mapper::Mapper::setRuntime ( hash  runtime)

adds runtime options to the current runtime option hash

Parameters
runtimea hash of runtime options to add to the current runtime option hash
Note
this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data
See also
Since
Mapper 1.1
Deprecated:
use Mapper construction options to set the runtime options.
Exceptions
RUNTIME-OPTION-CHANGEDthe option has changed during while already processing the data.

◆ validKeys()

hash Mapper::Mapper::validKeys ( )

returns a list of valid field keys for this class (can be overridden in subclasses)

Returns
a list of valid field keys for this class (can be overridden in subclasses)

◆ validTypes()

hash Mapper::Mapper::validTypes ( )

returns a list of valid field types for this class (can be overridden in subclasses)

Returns
a list of valid types for this class (can be overridden in subclasses)

Member Data Documentation

◆ m_empty_strings_to_nothing

bool Mapper::Mapper::m_empty_strings_to_nothing = False
private

flag to enforce deletion of the empty string in the output record

Since
Mapper 1.1

◆ m_runtime

*hash Mapper::Mapper::m_runtime
private

current runtime values

Since
Mapper 1.1