OpenDX Documentation
Full Contents QuickStart Guide User's Guide User's Reference
Previous Page Next Page Table of Contents Partial Table of Contents Index Search

5.3 Header File Syntax: Keyword Statements

The header of a General Array Importer file contains two or more of the keyword statements in the list that follows. The statements from block through type are listed in alphabetical order for convenient reference (they can be listed in any order in a header file). The placement of the first two statements in the list (file and grid|points) and the last two (positions and end) reflect syntactic requirements. The descriptions that follow this list occur in the same partly alphabetized order.

Notes:

  1. A statement or part of a statement enclosed in brackets ([ ]) is optional.
  2. Except for positions, no keyword statement can exceed one line in length.
  3. Any line beginning with a pound sign (#) is interpreted as a comment and ignored.
file = filename

grid = numx   x   numy   x   numz   x ...

or

points = n

[ block = skip1, elements1, width1, skip2, elements2, width2, ..., skipf, elementsf, widthf ]

[ dependency = dependency1, dependency2, ..., dependencyf ]

[ field = name1, name2, ..., namef ]

[ format = ( msb | lsb ] [ ascii | text | binary | ieee )]

[ header = ( bytes | lines | marker "string" )]

[ interleaving = ( field | record | record-vector | series-vector )]

[ layout = skip1, width1, skip2, width2, ..., skipf, widthf ]

[ majority = ( row | column )]

[ recordseparator = ( bytes n | lines n | marker "string" ), ( bytes n | lines n | marker "string" ), ... ]

[ series = t [ , start, delta ] [ , separator = ( bytes n | lines n | marker "string" ) ]]

[ structure = structure1, structure2, ..., structuref ]

[ type = type1, type2, ..., typef ]

[ positions = ( ( origin1, delta1, ..., origind, deltad ) | ( positiontype1, positiontype2, ..., positiontyped, position1, position2, ..., positiond ) | ( position1, position2, ..., positiond ) ) ]

[ end ]

file

file = filename
Function: Specifies the name of the file (including the path, if any) containing the data to be imported. The Importer searches the directory where the header file was found and any paths specified with the DXDATA environment variable (see the appropriate appendix in IBM Visualization Data Explorer User's Guide for information on environment variables).
Use: Required unless the header file contains an end statement. If this statement is omitted, the Importer assumes that the data are contained in the same file as the header and that they begin on the line immediately after the end statement or at the point specified by a header statement.

grid

grid = numx x numy x numz x ...
Function: Specifies the size and dimensions of the grid containing the data to be imported.
Use: Required unless the header file contains a points statement.

Notes:

  1. The num parameter is an integer specifying the number of coordinate points for a particular dimension (e.g., numx). The number of dimensions is implicitly specified by the number of such values provided. For example,
    grid = 2 x 2
    
    specifies a regular grid of 4 (four) points (the x between integers is required, but the blank spaces are optional.)
  2. The grid keyword (in contrast to points) implies connections between data points. An n-dimensional cuboid is assumed for connections. For example, a 1-dimensional grid generates a line connecting the positions.
  3. You can also specify that the number of grid elements are to be found in the data file. For the syntax, see B.1 , "General Array Importer: Keyword Information from Data Files" in IBM Visualization Data Explorer User's Guide.

points

points = n
Function: Specifies the number of data points to be imported.
Use: Required unless the header file contains a grid statement.

Notes:

  1. The points keyword (unlike grid) implies an absence of connections between data points.
  2. Unless the locations reserved word is used in the field keyword statement (see "field"), the positions are 1-dimensional. If this reserved word is not used, 1-dimensional positions can be specified with the positions keyword (see "positions"). Otherwise, positions are assumed to be regular, with an origin of 0 (zero) and a delta of 1 (one).
  3. You can also specify that the number of points is to be found in the data file. For the syntax, see B.1 , "General Array Importer: Keyword Information from Data Files" in IBM Visualization Data Explorer User's Guide.

block

[ block = skip1, elements1, width1, skip2, elements2, width2, ..., skipf, elementsf, widthf ]
Function: Specifies characteristics of each data field being imported. This keyword applies only to fixed-format ASCII data with record, record-vector, or series-vector interleaving.
Use: Optional.

Note: All three parameters take integer values. (Comma separators are optional.)

dependency

[ dependency = dependency1, dependency2, ..., dependencyf ]
Function: Specifies the dependencies of the data fields being imported.
Use: Optional. By default, data are assumed to be position dependent. Only if the header file also contains a grid statement (see "grid") may you specify connections (cell-centered) dependency (a points statement implies positions dependency).

Notes:

  1. For field-interleaved data, all fields must have the same dependency.
  2. The locations field (see "field Keyword Statement," Note 4) must be specified as position dependent.
  3. Comma separators are optional.

field

[ field = name1, name2, ..., namef ]
Function: Specifies the name and number of individual fields in a data file.
Use: Optional. If this keyword is not used, the number of fields is derived from other keywords (e.g., structure or type).

Notes:

  1. The values for the name parameter are strings separated by commas and without quotation marks.
  2. You must name all the fields in the data file.
  3. The field names can be used later to refer to individual fields. For example, if you use the Import module to import the data all at once, you can use the Select module to separate out each field by name. Or you can specify the names of the fields you want to import as the variable parameter to the Import module. For more information, see Import and Select in IBM Visualization Data Explorer User's Reference.
  4. If the position values are intermixed with the data in a file, you must specify the positions as a field (instead of using the positions keyword). Use the reserved word locations for the field name. The corresponding value for the structure keyword should specify the dimensionality of the positions (e.g., "2-vector").

format

[ format = ( msb | lsb ] [ ascii | text | binary | ieee )]

Function: Specifies the format and byte order of the data.
Use: Optional.

Notes:

  1. The accepted values for byte order are msb (most significant byte first) and lsb (least significant byte first). If the format is specified as binary, then the default byte order is the host byte order (i.e., the byte order of the machine on which the Import module is executing).
  2. For specifying the data format, ascii and text are synonymous, as are binary and ieee. The default is ascii. The supported binary form is IEEE.

header

[ header = ( bytes | lines | marker "string" )]

Function: Specifies how much material the Importer must skip before it begins reading data from a data file.
Use: Optional. By default, the Importer assumes that the data begin at the start of the file.
  • bytes n: the Importer will pass over n bytes (in a binary file) or n characters (in an ASCII file) before it begins to read data.
  • lines n: the Importer will pass over n lines before it begins to read data.
  • marker "string": the Importer will begin at the first character after the specified string. Quotation marks are required for a string containing blank spaces or commas.

Notes:

  1. If the data begins on the line following a marker, be sure to specify a new-line character ("\n") as part of the string. Note also that the spacing between the marker and the new-line character must be the same as that in the actual file. This spacing is easily determined in the Data Browser by placing the cursor at the end of the marker and then at the end of the line and reading the corresponding offset values (see Figure 18).
  2. If the marker itself contains quotation marks or special characters, use the escape character ("\") to indicate them, as shown in the following table.
    backslash                    \          \\
    backspace                    BS         \b
    bit pattern                  ddd        ddd
    carriage return              CR         \r
    double quote                 "          \"
    form feed                    FF         \f
    horizontal tab               HT         \t
    newline                      NL (LF)    \n
    Note: An octal value (ddd) can be used to specify special
          characters other than those shown here.
    

interleaving

[ interleaving = ( field | record | record-vector | series-vector )]

Function: Specifies to the Importer how the data in a data file are interleaved.
Use: Optional. By default, the Importer assumes that the interleaving is record-vector.

Note: The examples presented here are based on a 1-dimensional grid with 10 elements and two series members:

·
grid = 10
series = 2
field = t, v
structure = scalar, 3-vector
·
where t is a scalar value and v is a vector with three components (vx, vy, and vz). In the examples themselves, s represents a series member (0 or 1), and g represents a grid element number (0 through 9). The interleaving options are as follows:

field

Specifies column-oriented data such as that generated by a spreadsheet or by data-listing software. There is a separate "column" for each of the two fields with one element of each field per "row." (For non-scalar data, as here, there is a column for each vector component.) The number of rows corresponds to the size of the grid multiplied by the number of series members.


|- Field t -|--------------- Field v ---------------|
t(s=0,g=0),  vx(s=0,g=0),  vy(s=0,g=0),  vz(s=0,g=0),
t(s=0,g=1),  vx(s=0,g=1),  vy(s=0,g=1),  vz(s=0,g=1),
·
t(s=0,g=9),  vx(s=0,g=9),  vy(s=0,g=9),  vz(s=0,g=9),
t(s=1,g=0),  vx(s=1,g=0),  vy(s=1,g=0),  vz(s=1,g=0),
t(s=1,g=1),  vx(s=1,g=1),  vy(s=1,g=1),  vz(s=1,g=1),
·
t(s=1,g=9),  vx(s=1,g=9),  vy(s=1,g=9),  vz(s=1,g=9)

record

Specifies block or record-oriented data, where the values of all the elements of all the fields corresponding to one member (e.g., a time step) are listed before the elements and fields of the next member. For non-scalar fields, all the values of each vector component (e.g., all values of x) are listed in a separate record rather than in tuples (as they are in record-vector data; see below).
Note:For scalar fields, record and record-vector are the same.

t(s=0,g=0), t(s=0,g=1), ..., t(s=0,g=9),    ] Field t  *
vx(s=0,g=0), vx(s=0,g=1), ..., vx(s=0,g=9), *          |  Member
vy(s=0,g=0), vy(s=0,g=1), ..., vy(s=0,g=9), | Field v  |    s0
vz(s=0,g=0), vz(s=0,g=1), ..., vz(s=0,g=9), *          *
t(s=1,g=0), t(s=1,g=1), ..., t(s=1,g=9),    ] Field t  *
vx(s=1,g=0), vx(s=1,g=1), ..., vx(s=1,g=9), *          |  Member
vy(s=1,g=0), vy(s=1,g=1), ..., vy(s=1,g=9), | Field v  |    s1.
vz(s=1,g=0), vz(s=1,g=1), ..., vz(s=1,g=9)  *          *

The remaining two options (record- and series-vector) apply to cases in which vector components are stored together:

record-vector

Here the values of all the elements of all the components of all the fields corresponding to each member (e.g., a time step) are listed before those corresponding to the next member (e.g., all the data for s0 are listed first, followed by all the data for s1).

In addition, the components of each vector are stored as tuples (in contrast to the way they are stored in record data).

t(s=0,g=0), t(s=0,g=1), ..., t(s=0,g=9), ] Field t *
vx(s=0,g=0), vy(s=0,g=0), vz(s=0,g=0),   *         |
vx(s=0,g=1), vy(s=0,g=1), vz(s=0,g=1),   | Field   |  Member s0
...                                      |   v     |
vx(s=0,g=9), vy(s=0,g=9), vz(s=0,g=9),   *         *
t(s=1,g=0), t(s=1,g=1), ..., t(s=1,g=9), ] Field t *
vx(s=1,g=0), vy(s=1,g=0), vz(s=1,g=0),   *         |
vx(s=1,g=1), vy(s=1,g=1), vz(s=1,g=1),   | Field   |  Member s1
...                                      |   v     |
vx(s=1,g=9), vy(s=1,g=9), vz(s=1,g=9)    *         *

series-vector

Here the values of all the elements of all the members (e.g., time steps) are listed for one field before those of the next field (e.g., all the data for field t are listed first, followed by all the data for field v).

In addition, the components of a vector are stored as tuples rather than in separate records (as they are stored in record data; see above).

t(s=0,g=0), t(s=0,g=1), ..., t(s=0,g=9),  * Field  ] Member s0
t(s=1,g=0), t(s=1,g=1), ..., t(s=1,g=9),  *   t    ] Member s1
vx(s=0,g=0), vy(s=0,g=0), vz(s=0,g=0),    *        *
vx(s=0,g=1), vy(s=0,g=1), vz(s=0,g=1),    |        | Member
...                                       |        |   s0
vx(s=0,g=9), vy(s=0,g=9), vz(s=0,g=9),    | Field  *
vx(s=1,g=0), vy(s=1,g=0), vz(s=1,g=0),    |   v    *
vx(s=1,g=1), vy(s=1,g=1), vz(s=1,g=1),    |        | Member
...                                       |        |   s1
vx(s=1,g=9), vy(s=1,g=9), vz(s=1,g=9)     *        *

layout

[ layout = skip1, width1, skip2, width2, ..., skipf, widthf ]

Function: Specifies the number of bytes (characters) the Importer must skip before it begins to read a field's data and then the number of bytes it should read (i.e., the "width" of the data item).
Use: Optional. This keyword applies only to ASCII, field-interleaved data. If the data is in ASCII format but the keyword is not used, the Importer assumes a default of one or more blank spaces (space, tab, new line, or form feed) as the delimiter between fields.

Notes:

  1. The components of a vector must each be represented in the same number of characters. Comma separators are optional.

    The following statement tells the Importer to skip 10 characters, read one field of 6 characters, skip 10 characters, and read another field of 4 characters.

    layout = 10, 6, 10, 4
    
  2. Because skip specifies the number of characters to be passed over before each field, it does not apply to single elements of a vector field but to the field as a whole. However, width applies to each element of the vector. For example, the following statement tells the Importer to skip 10 characters, then read a field 6 characters long:
    layout = 10, 6
    
    If the field v is a 2-vector, then each component (vx and vy) is 6 characters long, for a total width of 12 characters.
  3. If there are no spaces between two fields, specify the skip value as 0 (zero).

majority

[ majority = ( row | column )]

Function: Specifies the organization of multidimensional arrays composing a data field.
Use: Optional. The default is row (last dimension varies fastest, as in the C programming language). Column majority means that the first dimension varies fastest, as in the FORTRAN programming language.

Note: The maximum number of dimensions supported for column majority is 4.

recordseparator

[ recordseparator = ( bytes n | lines n | marker "string" ), ( bytes n | lines n | marker "string" ), ... ]

Function: Specifies the separation between records.
Use: Optional. This keyword applies only to record and record-vector interleaving.

Notes:

  1. With differences to be noted here, the specification of separation is very much like that of the header keyword (see "header").
  2. If all records are separated by the same amount, a single separator value should be specified. For example, if each pair of successive records is separated by two lines, then
    recordseparator = lines 2
    
  3. If the records are separated by different amounts, a value must be specified for each of the separators:

series

[ series = t [ , start, delta ] [ , separator = ( bytes n | lines n | marker "string" ) ]]

Function:

Specifies, to the Importer, information about a series.

Use:

Optional. This keyword is required only for the importation of series data. The default assumes no separation between series sections. Descriptive information between series sections is described with the separator parameter. If descriptive information precedes the first member of the series, it can be skipped by means of a header statement.

  • t is a required parameter that specifies the number of series elements in the data file. Its default value is 1.
  • start and delta are optional parameters that specify the series positions. The position values are defined as: < start, start plus d e l t a, start plus (2 times d e l t a), ellip, start plus (t minus 1) times d e l t a>

    The defaults for start and delta are 0 (zero) and 1 (one) respectively.

  • The following example specifies that there are four series members, with positions 0.8, 1.2, 1.6, and 2.0.
    series = 4, 0.8, 0.4
    
  • The specification of separator is very much like that of the header keyword (see "header").

structure

[ structure = structure1, structure2, ..., structuref ]

Function: Specifies the structure of each field in a data file.
Use: Optional. The default is scalar. Accepted values are scalar, string[ n ], and 2-vector , ..., 9-vector for each field. However, 5-vector, ..., 9-vector cannot be specified for column-majority arrays. In string[ n ], n specifies the length of the longest string.

Notes:

  1. Use of this keyword requires specifying the structure of all fields in the file.
  2. The following example specifies that the first two fields have a scalar structure, while the third is a vector with three components:
    structure = scalar, scalar, 3-vector
    
  3. Since the default is scalar, the statement is not required if, say, six scalar fields are to be imported. But if one of these fields is vector, the statement is required; for example:
    structure = scalar, scalar, 3-vector, scalar, scalar, scalar
    
  4. If string data contain embedded blanks, you must use the layout or block keyword to specify how the string is to be read.

type

[ type = type1, type2, ..., typef ]

Function: Specifies the data type for each specified field.
Use: Optional. The default is float. The accepted values are:

double      byte               int               short
float       signed byte        signed int        signed short
string      unsigned byte      unsigned int      unsigned short

Notes:

  1. Use of this keyword requires specifying the type for all fields in the file.
  2. The following are pairs of equivalent types:

positions

[ positions = ( ( origin1, delta1, ..., origind, deltad ) | ( positiontype1, positiontype2, ..., positiontyped, position1, position2, ..., positiond ) | ( position1, position2, ..., positiond ) ) ]

Each syntax line in this diagram is discussed separately in Note 4 below.
Function: Defines the positions component of the fields in a data file.
Use: Optional. The default is regular positions in compact notation, where the origin and delta in each dimension are 0.0 and 1.0 respectively, unless the locations keyword has been used.

Notes:

  1. This keyword must be placed at the end of the header file or immediately preceding the end keyword.
  2. The numbers specified in a positions statement can span any number of uninterrupted lines (including carriage returns, which is not true of other keyword statements).
  3. Positions can also be specified with the field statement and the locations reserved word (see "field"). This alternative is important in cases where:
  4. There are four ways to specify positions with the positions keyword:

end

[ end ]
Function: Causes the Importer to stop processing header statements.
Use: Optional unless the data are in the same file with the header statements. By default, the Importer stops processing at the end of the header file.


Full Contents QuickStart Guide User's Guide User's Reference

[ OpenDX Home at IBM | OpenDX.org ]