 |
Qore Mapper Module Reference
1.5.3
|
|
Qore Mapper Module Reference
1.5.3
|
This module provides classes that help with structured data mapping, meaning the transformation of data in one or more input formats to a different output format.
Classes provided by this module:
The following is an example map hash with comments:
If this map is applied to the following data in the following way:
The result will be:
list: (2 elements)
[0]=hash: (5 members)
id : 1
name : "John Smith"
explicit_count : 1
implicit_count : 1
order_date : 2014-01-02 10:37:45.103948 Thu +01:00 (CET)
[1]=hash: (5 members)
id : 2
name : "Steve Austin"
explicit_count : 2
implicit_count : 2
order_date : 2014-01-04 19:21:08.882634 Sat +01:00 (CET))The mapper hash is made up of target (ie output) field names (note that dotted output field names result in a nested hash output unless the allow_output_dot option is set) as the key values assigned to field specifications as follows:
"name" key)"code" key); the closure or call reference must accept the following arguments:
"code": a closure or call reference to process the field data; cannot be used with the "constant" or "index" keys"constant": the value of this key will be returned as a constant value; this key cannot be used with the "name", "struct", "code", "index" or "default" keys"index": gives current index/count of the row. The initial int value is the start offset. So value 0 means that mapped values will be: 0, 1, ..., N; 1 means: 1, 2, ..., N; etc."date_format": gives the format for converting an input string to a date; see Date Formatting Codes for the format of this string; note that this also implies "type" = "date" "default": gives a default value for the field in case no input or translated value is provided"mand": assign to boolean True if the field is mandatory and an exception should be thrown if no input data is supplied"maxlen": an integer giving the maximum output string field length in bytes"name": the value of this key gives the name of the input field; only use this if the input record name is different than the output field name; note that if this value contains "." characters and the allow_dot option is not set (see Mapper Options), then the value will be treated like "struct" (the "struct" key value will be created automatically); cannot be used with the "constant" ior "index" keys"number_format": gives the format for converting an input string to a number; see Qore::parse_number() for the format of this string; note that this also implies "type" = "number" "output_key_path": gives the output path for hash output values; each element in the list is a string key name"runtime": a reference to Mapper Runtime Options current status. The value is key in the current runtime structure."struct": the value of this key gives the location of the input field in an input hash in dot notation, ex: "element.name" would look for the field's value in the "name" key of the "element" hash in the input record; cannot be used with the "constant" or "index" keys; this option is only necessary in place of the "name" option if the allow_dot option is set, otherwise use "name" instead"trunc": assign to boolean True if the field should be truncated if over the maximum field length; this key can only be set to True if the "maxlen" key is also given"type": this gives the output field type, can be:"date": date/time field"int": fields accepts only integer values (any non-integer values on input will cause an exception to be thrown when mapping; note: also "integer" is accepted as an alias for "int")"number": field accepts only numeric values (any non-numeric values on input will cause an exception to be thrown when mapping); numeric values are left in their original types, any other type is converted to a arbitrary-precision numeric value"string": field accepts string values; in this case any other value will be converted to a string in the output"hash": field accepts hash values"any": field accepts any value"type_options": a hash of type options to set or override type options for the output fieldMapper objects accept the following options in the option hash:
"info_log": an optional info logging callback; must accept a string format specifier and sprintf()-style arguments"input": this should be a description of the input fields with type hash<string, DataProvider::AbstractDataField>, for backwards compatibility, this option also accepts a hash describing the input fields where each key is a possible input field name (and where dot notation indicates a multi-level hash) and each value is a hash describing the field with the following optional keys:"desc": this gives the description of the input field"input_log": an optional input data logging callback; must accept a hash giving the input data hash"input_provider": gives the input provider with an AbstractDataProvider object which defines the type of input data and also the data itself. The use of this option enables the use of the Mapper::getOutputIterator() API. This option is mutually exclusive with the input option. If an "output_provider" is also provided, the Mapper::runAutonomous() method can be used to map from input to output in a single call"input_provider_search": the search criteria for the input provider; see the where_cond option of AbstractDataProvider::searchRecords() for more information on this option"input_request": the arguments for input providers using the request/response API"input_request_options": any options to input providers using the request/response API"input_response_error": a string indicating the input providers using the request/response API should use the given error response message for the record format"input_search_options": the search options for the input provider; see the search_options option of AbstractDataProvider::searchRecords() for more information on this option"name": the name of the mapper for use in logging and error strings"output": this should be a description of the output fields with type hash<string, DataProvider::AbstractDataField>, for backwards compatibility, this option also accepts a hash describing the output data structure where each hash key is a output field name (and where dot notation indicates a multi-level hash) and each value is an optional hash describing the output field taking a subset of mapper field hash keys as follows:"desc": a description of the output field"mand": True if the field is mandatory and an exception should be thrown if no input data is supplied"maxlen": an integer giving the maximum length of a string field in bytes"type": this gives the output field type, can be:"date": date/time field"int": fields accepts only integer values (any non-integer values on input will cause an exception to be thrown when mapping; note: also "integer" is accepted as an alias for "int")"number": field accepts only numeric values (any non-numeric values on input will cause an exception to be thrown when mapping); numeric values are left in their original types, any other type is converted to a arbitrary-precision numeric value"string": field accepts string values; in this case any other value will be converted to a string in the output"hash": field accepts hash values"any": field accepts any value"output_log": an optional output data logging callback; must accept a hash giving the output data hash"output_nullable": set all output fields as nullable"output_provider": gives the output provider with an AbstractDataProvider object which defines the type of output data and also location where the output data will be written. If this option is set, then every mapped record will be written to the output data provider automatically."input_provider" is also provided, the Mapper::runAutonomous() method can be used to map from input to output in a single call"output_provider_bulk": if this option is used with a record-based output provider, then bulk operations are used with the output provider, and the Mapper::flushOutput() method must be called after all mapping is done to flush the output buffer to the output provider at the end, or Mapper::discardOutput() must be called to discard any data left in the bulk output buffer if the results should be discarded (ex: the output provide requires transaction management and an error occurs causing the transaction to be rolled back)"output_provider_upsert": set to True if upsert operations instead of creation APIs should be used with the output provider. If output_provider_bulk is also set, this indicates if the AbstractDataProviderBulkOperation object will use upsert operations instead of insert operations"runtime": an initial runtime structure for Mapper Runtime Options"trunc_all": if True (as evaluated by parse_boolean()) then any field without a "trunc" key (see Mapper Specification Format "trunc" description) will automatically be truncated if a "maxlen" attribute is set for the fieldThe following deprecated options are also accepted:
"allow_dot": if True (as evaluated by parse_boolean()) then field names with "." characters do not imply a structured internal element lookup; in this case input field names may have "." characters in them, use the "struct" key to use structured internal element loopups (see Mapper Specification Format "struct" docs for more info)"allow_output_dot": if True (as evaluated by parse_boolean()) then output field names with "." characters do not imply a structured/hash output element; in this case output field names may have "." characters in them"date_format": gives the global format for converting a string to a date; see Date Formatting Codes for the format of this string; this is applied to all fields of type "date" unless the field has a "date_format" value that overrides this global setting"encoding": the output character encoding; if not present then "UTF-8" is assumed"input_timezone": an optional string or integer (giving seconds east of UTC) giving the time zone for parsing input data (ex: "Europe/Prague"), if not set defaults to the current TimeZone (see Qore::TimeZone::get())"number_format": gives the global format for converting a string to a number; see Qore::parse_number() for the format of this string; this is applied to all fields of type "number" unless the field has a "number_format" value that overrides this global setting"timezone": an optional string or integer (giving seconds east of UTC) giving the time zone definition for output data (ex: "Europe/Prague"), if not set defaults to the current TimeZone (see Qore::TimeZone::get())"input" option is given, then only those defined fields can be referenced as input fields in the mapper hash; all possible input fields should be defined here if this option is used"output" option is given, then only those defined fields can be referenced as output fields, additionally the types given in the output definition cannot be overridden in the mapper hash; all possible output fields should be defined here if this option is usedRuntime options for Mapper objects allow the programmer to use constant values provided at runtime in the Mapper output.
For example, runtime options can be useful in the following cases:
The runtime options are basically the same as setting constants in the mapper before providing runtime data to the mapper. As such, the runtime options can be changed only before the first input hash is processed by a Mapper.
Note that the Mapper::setRuntime() and Mapper::replaceRuntime() methods are deprecated - please use Mapper construction options to set runtime values instead. The methods are deprecated since runtime options duplicate existing functionality and are confusing and error-prone to use.
output_create_ignore_duplicates optionsubmappers option (issue 3414)ignore_missing_input mapper option (issue 3837)use_input_record output field mapping key (issue 3823)output_nullable option (issue 3788)STRING-TOO-LONG exception (issue 2405)TableMapper module (issue 1736)"constant" field tag giving a constant value for the output of a field"allow_output_dot" option to suppress this behavior"default" field tag giving a default value if no input value is specified"date_format" mapper option"number_format" field option and a global option of the same name"timezone" and "input_timezone" options, documented those options"number" field type: now leaves numeric values in their original type, converts all other types to a number"crec" option"input" option with input record validation"output" option with output record validation"info_log" option and removed the "trunc" option"runtime" mapper option"index" field tag for current row index