Qore CsvUtil Module Reference  1.10
CsvUtil::AbstractCsvWriter Class Referenceabstract

The AbstractCsvWriter class provides a parent for all CSV writers. More...

Inheritance diagram for CsvUtil::AbstractCsvWriter:

Public Member Methods

 constructor (string n_errname, *hash< auto > n_opts)
 Creates the AbstractCsvWriter in single-type mode. More...
 
 constructor (string n_errname, hash< auto > spec, hash< auto > n_opts)
 Creates the AbstractCsvWriter in single-type mode. More...
 
*hash< string, AbstractDataField > getRecordType ()
 Returns the description of the record type, if any.
 
 write (list< auto > l)
 Stream the contents of the list into the output. More...
 
 write (Qore::AbstractIterator iterator)
 Stream an iterator into the output. More...
 
 write (Qore::SQL::SQLStatement iterator)
 Stream an iterator into the output. More...
 
 writeLine (hash< auto > values)
 Write a line with headers-values hash. More...
 
 writeLine (list< auto > values)
 Write a line with a list of values; data are checked against column rules. More...
 
 writeLine (string type, hash< auto > values)
 Write a line for a specific record from a hash to the output. More...
 
 writeLine (string type, list< auto > values)
 Write a line with headers-values list. More...
 

Private Member Methods

string prepareRawLine (list< auto > values)
 Prepare a string (line with EOF) with formatting and escaping. More...
 
 processCommonOptions (*hash< auto > n_opts, int C_OPTx)
 Process options and set internal variables.
 
 processSpec ()
 Process specification and set internal variable for mapping.
 
 writeHeaders ()
 Write csv headers.
 
abstract writeRawLine (list< auto > values)
 This method must be overridden in child classes to provide the output implementation.
 

Private Attributes

string baseTemplate
 base template for value format
 
int block = 1000
 block size for bulk DML
 
bool checkElementCounts = False
 verify the column count for every row; if a row does not match, then throw a CSVFILEITERATOR-DATA-ERROR exception
 
string encoding
 output file character encoding
 
string eol = EOL_UNIX
 end of line sequence
 
*code info_log
 a closure/call reference for informational logging when using write(SQLStatement)
 
int lineNo = 0
 the latest line number
 
hash m_out_by_idx
 mapping output field by index
 
hash m_out_by_name
 mapping output field by name
 
string m_quoteEscapeChar = "\\"
 quote escape character
 
bool optimal_quotes = True
 stores the optimal quotes option
 
const Options = ...
 valid options for the object (a hash for quick lookups of valid keys)
 
string quote = "\""
 field content delimiter
 
string separator = ","
 field separator
 
bool write_headers = True
 this flag determines if any stored headers are output
 

Detailed Description

The AbstractCsvWriter class provides a parent for all CSV writers.

Any inherited class must provide concrete implementations for the following abstract method, where the real physical write action must be implemented.

private writeRawLine(list values)

AbstractCsvWriter Constructor Option Hash Overview

The AbstractCsvWriter class constructor takes an optional hash with possible keys given in the following table. Note that key names are case-sensitive, and data types are soft (conversions are made when possible).

AbstractCsvWriter Options

Option Data Type Description
"block" int the block size when generating output based on a SQLStatement source (default: 1000)
"date_format" string the default mask for date values formatting
"encoding" string the character encoding for the file (and for tagging string data read); if the value of this key is not a string then it will be ignored
"eol" string the end of line character(s) (default: auto-detect); if the value of this key is not a string then it will be ignored
"info_log" code a call reference / closure for informational logging when performing bulk generation with write(Qore::SQL::SQLStatement); must take a single string argument
"optimal_quotes" bool set to False to disable optimal quoting; when optimal quoting is disabled, all fields are quoted regardless of type or content, when it is enabled, then fields are quoted only if they require quoting (i.e. they contain a quote or separator character); the default is True
"quote" string the field quote character (default: '"')
"quote_escape" string the escape character(s) used for "quote" (default: '\')
"separator" string the string separating the fields in the file (default: ",")
"verify_columns" bool if True (the default is False) then if a line is parsed with a different column count than other lines, a CSVFILEWRITER-DATA-ERROR exception is thrown
"write_headers" bool set to False to suppress the output of headers; the default is True, meaning to output headers if they are present. The value is False for multi-type lines.

AbstractCsvIterator Single-type-only Options

Option Data Type Description
"datamap" hash a hash mapping actual data key names to the output field names, for use in case the data field names differ; does not have to include every data or output key; keys not present will be assumed to be mapped 1:1
"fields" hash describes the data to be output; see option field hash and option field types for more information on the possible values of this option; if no field type is given, the default is "*string"
"headers" list of strings list of header / column names for the data iterated.
"header_reorder" bool if True (default value) then if "headers" are provided by options then fields are reordered to follow headers. It has effect on expected field order as passed to write function via list. If False then fields not yet specified are pushed at the end of field definition.
Note
the following options with dashes in their names are supported for backwards-compatibility:
  • "date-format"
  • "optimal-quotes"
  • "verify-columns"
  • "write-headers"

Member Function Documentation

◆ constructor() [1/2]

CsvUtil::AbstractCsvWriter::constructor ( string  n_errname,
*hash< auto >  n_opts 
)

Creates the AbstractCsvWriter in single-type mode.

Parameters
n_errnamea string to construct child class error message. For example: value 'FOO' will result in exception names 'CSVFOOWRITER-ERROR'
n_optsAbstractCsvWriter Constructor Option Hash Overview
Exceptions
CSV<errname>WRITER-ERRORin the case of incorrect options

◆ constructor() [2/2]

CsvUtil::AbstractCsvWriter::constructor ( string  n_errname,
hash< auto >  spec,
hash< auto >  n_opts 
)

Creates the AbstractCsvWriter in single-type mode.

Parameters
speca hash of field and type definition; see Option Field Hash for more information
n_errnamea string to construct child class error message. For example: value 'FOO' will result in exception names 'CSVFOOWRITER-ERROR'
n_optsAbstractCsvWriter Constructor Option Hash Overview
Exceptions
CSV<errname>WRITER-ERRORin the case of incorrect options

◆ prepareRawLine()

string CsvUtil::AbstractCsvWriter::prepareRawLine ( list< auto >  values)
private

Prepare a string (line with EOF) with formatting and escaping.

Parameters
valuesa list with values
Returns
string a formatted and escaped line with a trailing EOL string; if an encoding option was given to the constructor; the return value always has the given encoding

◆ write() [1/3]

CsvUtil::AbstractCsvWriter::write ( list< auto >  l)

Stream the contents of the list into the output.

Parameters
la list of input data to format as CSV output

The list has to contain List or Hash elements that can be formatted according to the CSV definition

Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints or when the list elements have the wrong type or values

◆ write() [2/3]

CsvUtil::AbstractCsvWriter::write ( Qore::AbstractIterator  iterator)

Stream an iterator into the output.

Parameters
iteratoran iterator to stream data into file

The iterator has to return List or Hash from the getValue() method.

Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints or when getValue() does not return list or hash

◆ write() [3/3]

CsvUtil::AbstractCsvWriter::write ( Qore::SQL::SQLStatement  iterator)

Stream an iterator into the output.

Parameters
iteratoran SQLStatement iterator to stream data into file; SQLStatement::fetchColumns() is used to leverage bulk DML for more efficient SQL I/O
Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints
Note
if any "info_log" option is set in the constructor; it is used here to log each block of SQL data used to generate the corresponding number of lines; log messages look like: "query input generated bulk output lines: 1000"

◆ writeLine() [1/4]

CsvUtil::AbstractCsvWriter::writeLine ( hash< auto >  values)

Write a line with headers-values hash.

Parameters
valuesa hash with keys as column headers in single-type format or with hash with "type" and "record" members for multi-type. The format is automatically detected.
Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints

◆ writeLine() [2/4]

CsvUtil::AbstractCsvWriter::writeLine ( list< auto >  values)

Write a line with a list of values; data are checked against column rules.

Parameters
valuesa list with values.
Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints

◆ writeLine() [3/4]

CsvUtil::AbstractCsvWriter::writeLine ( string  type,
hash< auto >  values 
)

Write a line for a specific record from a hash to the output.

Parameters
typerecord type
valuesa hash of values.
Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints

◆ writeLine() [4/4]

CsvUtil::AbstractCsvWriter::writeLine ( string  type,
list< auto >  values 
)

Write a line with headers-values list.

Parameters
typerecord type
valuesas list of values
Exceptions
CSVFILEWRITER-DATA-ERRORwhen the data does not fit defined column constraints