Aggregate Token Length
(Text Processing)
Synopsis
Extracts and aggregates the lengths of all tokens of a document. The result will be added as new meta data.Description
This operator iterates over all tokens of a given document and aggregates the length of each token with the aggregation function specified by the user. The resulting value will be added as a numerical meta data entry. The name of the entry can be defined by the user.
Input
documentThe document port.
Output
- document
The document port.
Parameters
- metadata keyThe aggregated value of the tokens' length will be saved under this meta data key. The key will become the name of the attribute after document processing.
- aggregationThis specifies how the tokens' lengths are aggregated to one single number.