Build a deduplication workflow
Deduplicate four sample rows locally and compare the result with the expected rows.
This example uses the local Data transform and a test project. It needs no external capability or source file.
- Choose Workflows → Start delivery → Create blank flow. If the draft opens with example read and delivery steps, remove them. Add Data transform. If you keep the Goal / brief step, connect it to the transform.
- In Workflow → Start Form / Run Input, replace the default
topicfield with a required array namedrows. - Select the transform. In its input mappings, add a target
itemspointing to$.run.input.rows. If you want the mapping in the reference picker, declareitemsas an array in the step’s expected input fields. - Set the processing method to Remove duplicates, key to
id, input source to/items, and output name touniqueRows. The transform spec under the step’s advanced settings should have these values.
{
"schemaVersion": "raydo.data_transform.v1",
"operation": "dedupe",
"source": "/items",
"key": "id",
"outputKey": "uniqueRows"
}- Use Review & Publish and complete the release reason, approval comment, and approver signature to activate the flow in the test workspace.
- Choose Start Run, select the test project, and enter the following run input.
{
"rows": [
{"id": "A01", "name": "Sample A"},
{"id": "A01", "name": "Duplicate A"},
{"id": "B02", "name": "Sample B"},
{"id": "B02", "name": "Duplicate B"}
]
}In Runs & Approvals, open the transform’s step output. result.uniqueRows should contain Sample A and Sample B. Deduplication keeps the first occurrence of each key. To use the array in a downstream step, map $.nodes.<actual-transform-node-id>.output.result.uniqueRows after connecting the steps. A missing key can cause rows to collapse together, so inspect the upstream rows before deploying the transform.
See the data processing pattern and data and flow nodes.