PDI 1.7.1

the PDI data interface

Set_value plugin

The Set_value plugin allows setting values to data and metadata descriptors from the yaml file.

Here are main features of the plugins:

  1. Share data - plugin will share new allocated data with given values
  2. Release data - plugin will release shared data
  3. Expose data - plugin will expose new allocated data with given values
  4. Set data - plugin will set given values to the already shared data
  5. Calling an event - plugin will call an event

Configuration grammar

At its root, the Set_value configuration is made of:

key value
"logging" (optional) a logging
"on_init" (optional) a list of value operation
"on_event" (optional) an on_event
"on_data" (optional) an on_data
"on_finalize" (optional) a list of value operation

on_init

Specifies a list of operation to do in PDI_init function. on_init is a list of value operation s.

plugins:
set_value:
on_init:
- set: ... # value_list
- share: ... # value_list
- expose: ... # value_list
- release: ... # value_list
- event: ... # value
- logger: ... # logger_map

on_event

Specifies a map of events that will trigger the values set. on_event is a map with event name as a key and value operation list as map value.

key value
".*" (optional) a list of value operation
plugins:
set_value:
on_event:
event_1_name:
- set: ... # value_list
- share: ... # value_list
- expose: ... # value_list
- release: ... # value_list
- event: ... # value
event_2_name:
- set: ... # value_list
- share: ... # value_list
- expose: ... # value_list
- release: ... # value_list
- event: ... # value

on_data

Specifies a map of data that on share will trigger the values set. on_data is a map with data/metadata name as a key and value operation as map value.

key value
".*" (optional) a list of value operation
metadata:
metadata_1_name: ... # metadata_type
data:
data_1_name: ... # data_type
plugins:
set_value:
on_data:
data_1_name:
- set: ... # value_list
- share: ... # value_list
- expose: ... # value_list
- release: ... # value_list
- event: ... # value
metadata_1_name:
- set: ... # value_list
- share: ... # value_list
- expose: ... # value_list
- release: ... # value_list
- event: ... # value

on_finalize

Specifies a list of operation to do in PDI_finalize function. on_finalize is a list of value operation s.

plugins:
set_value:
on_finalize:
- set: ... # value_list
- share: ... # value_list
- expose: ... # value_list
- release: ... # value_list
- event: ... # value

value operation

A value operation is specified as a key-value pair (a mapping whose content is a single key-value pair). It defines an operation to execute inside a value list.

key value
"set" (optional) a value list
"share" (optional) a value list
"expose" (optional) a value list
"release" (optional) a list with data to release
"event" (optional) an event name to call
"logger" (optional) a logger map

Share warning

Share is always done with read and write rights. Plugin allocates memory by malloc. If you reclaim the data, you should free it with free instruction.

value list

Is a list of the

as elements. The order of the elements is important. The descriptors will be set in the same order as given in the yaml file.

scalar value

Is a map of descriptor name as a key, and a value to set as a value:

key value
".*" (optional) a scalar value to set
metadata:
scalar_name: int
set_value:
on_init:
- expose:
- scalar_name: 42

array value

Is a map of descriptor name as a key, and a list of values to set as a value:

key value
".*" (optional) a list of value to set
metadata:
array_name:
type: array
size: 4
subtype: int
plugins:
set_value:
on_init:
- expose:
- array_name: [2, 3, 4, 5]

record value

Is a map of descriptor name as a key, and a list of members to set as a value. Each member consist of an inner value_list. The order of the members is unrestricted, but the plugin will set the values to the members in the same order.

key value
".*" (optional) a map with member name as key and a value to set as value
metadata:
record_name:
type: record
buffersize: 16
members:
member_1:
disp: 0
type: array
size: 3
subtype: int
member_2:
disp: 12
type: int
set_value:
on_init:
- expose:
- record_name:
member_2: 3 # int member
member_1: [1, 2, 3] # array of ints member

logger map

Defines settings for global PDI logger.

key value
"level" (optional) level to set to the logger
"pattern" (optional) pattern to set to the logger
"evaluate" (optional) if true evaluate the logger pattern

full configuration example

metadata:
array_size: int64
data:
record_data:
type: record
buffersize: 36
members:
scalar_data:
disp: 0
type: int
array_data:
disp: 4
type: array
size: $array_size
subtype: int
plugins:
set_value:
on_init:
- expose:
- array_size: 3
- share:
- record_data:
- scalar_data: 0
- array_data: [0, 0, 0]
on_event:
event_1_name:
- set:
- record_data:
- scalar_data: 3
- array_data: [1, 2, 3]
on_finalize:
- release: [record_data]

Using old values to set new

Increment value

The set_value plugin allows to use the old values to set new values, you can use even the same descriptor, for example to increment a scalar.

data:
value_int:
type: int
int_array:
size: 3
subtype: int
type: array
plugins:
set_value:
on_event:
init:
- share:
- value_int: 0
- int_array: [1, 2, 3]
increment:
- set:
- value_int: "$value_int + 1"
- int_array: ["$int_array[0] + 1", "$int_array[1] + 1", "$int_array[2] + 1"]
on_finalize:
- release: [value_int, int_array]

After calling init and increment event, value_int will be equal 1, and int_array to [2, 3, 4].

Getting old value

The new value is set at the end of processing the whole descriptors. This means, that if you want to update the array element depending on other element, the old value will be set:

metadata:
int_array:
size: 3
subtype: int
type: array
plugins:
set_value:
on_event:
init:
- expose:
- int_array: [0, 0, 0]
increment:
- expose:
- int_array: ["$int_array[0] + 1", "$int_array[0] + 1", "$int_array[1] + 1"]

After calling init and increment event, all values in int_array will equal 1. This is because the int_array[0] was updated after setting all the elements.

Getting new value

metadata:
int_scalar: int
int_array:
size: 3
subtype: int
type: array
plugins:
set_value:
on_event:
init:
- expose:
- int_scalar: 0
- int_array: [0, 0, 0]
increment:
- expose:
- int_scalar: $int_scalar+1
- int_array: ["$int_scalar", "$int_scalar", "$int_scalar"]

After calling init and increment event, all values in int_array will equal 1. This is because the int_scalar is set and then the int_array is updated after the int_scalar has a new value.