Transform records (Data Transformation)
Use the Quick Processor to modify existing attributes of your data objects or add new ones that can be used for filtering using the attribute filters or sorting.
In combination with the Multi-Select functionality, you can use the Quick Processor to correct data, select it via Multi-Select and send it to a stream again. See the following video to learn how to correct records using the Quick Processor.
Calculation of results based on multiple data sets (e.g., moving average or grouping)
Use the state store of the Quick Processor to store calculation results across multiple executions (e.g., to calculate a moving average) or to group data sets.
Complex filter logic
This code randomly picks (filters) a record out of your stream and sets the record's value to "Hello World!".
if ( Math.random() < 0.5) // Filter a record randomly.
return false; // If false, the record is not displayed.
rec.value = "Hello World!"; // Sets a new value for each record
return rec; // Returns the new record with "Hello World!".
Overwrite, delete or calculate new values for message value attributes and keys.
// Overwrite the record's key and add a new attribute
// to the record's value.
rec.value.myAttribute = "Hello World!";
rec.key = "My Key";
// Delete an attribute
// No filtering (all records are let through)
// View only records that are from January (0)
return new Date(rec.value.deliveryTime).getMonth() != 0;
Store data (aggregates)
Calculate and temporarily store data to aggregate or group data.
// If the value or key is a JSON object with a field "myAttr":
Tip: Quickly copy the path of an attribute to the clipboard
Writing out the path of an attribute can be annoying. Therefore, select a record in the record list and click on the corresponding attribute in the detail view. This opens the filter bubble. Click on the name of the attribute on the left. You now have the complete path in the clipboard.
State and stateful calculations
The Quick Processor is executed for each individual data record. To group records or perform calculations based on a previous data set or result, a state can be saved in the state store.
A state is saved with the store(any) command. The restore():any command can be used to load the state from the state memory into a variable, for example. To initialize a state memory (e.g. to set an initial value), the command hasStore():boolean can be used to query whether a state already exists.
Moving average example
This code gets the value of the record and adds it to the value of the previous record that was stored. It then divides the result by two to get the moving average and returns a JSON object with the moving average and the current temperature.
var fahrenheit = rec.value;
store(fahrenheit); // Init state with current temperature
// Divide the previous average by 2;
store((restore() + fahrenheit) / 2);
// Setting the calculated and the original value as the new record's value
The following diagram illustrates the data flow of a data set. The Quick Processor is executed after the decoding process for each individual data set. A Quick Processor can be used for data transformation, but can also act as a filter with complex ...
Data protection is a crucial aspect of business operations, especially when dealing with sensitive information. Kadeck Teams Enterprise is equipped with a Data Protection Policy module that helps you define data protection policies through the ...
To modify or create a Quick Processor, the user must have both the QuickProcessorModify right and the TopicAccessRead right. QuickProcessorModify can be assigned for individual connections/environments. In combination with the TopicAccessRead right, ...
This article gives a brief introduction of custom codecs in Kadeck Desktop and Kadeck Teams and covers the essential steps to get you up and running quickly. It is recommended to read Structure of Codecs before starting developing your own codec as ...
We are excited to introduce Kadeck's new Monitoring Module, which makes monitoring and troubleshooting Apache Kafka infrastructure easier than ever and raises the bar in the industry. The new monitoring module includes four new pages, Monitoring ...