How to Use Merge Task Within a Playbook

LAST UPDATED: 09/25/2023

Overview

A Merge Task is used to merge execution paths based on a condition. When the merge occurs, the execution paths or the specified data are grouped into an array and saved in the rawData field for the task instance where the merge occurs, and tasks linked from the Merge Task are initiated.

Input Parameters

Condition (JSON Object)

Condition Syntax

{
    "groupBy": "<Playbook Instance|Task|Task Instance|Path>",
    "taskName": "<Task Name>",
    "paths": ["<JSON Path 1>", "<JSON Path 2>", ...],
    "dataJsonPath": "<JSON Path>",
    "count": <Number>
}

Key Definition

READER NOTE

Set dataJsonPath to merge specific data only to improve performance. Otherwise, all the execution paths in the same group are merged on the instance of the Merge task where the merge occurs, and it could generate large data and consequently affect performance.

Group by Playbook Instance

All the execution paths are grouped into one group.

Sample 1: The following playbook demonstrates how to merge all the execution paths by a Merged By Playbook Instance task.

Condition

{
    "groupBy": "Playbook Instance",
    "dataJsonPath": "$['Process Name'].returnData"
}

Description

1. There are 8 execution paths arriving at the Merged By Playbook Instance task, resulting in the creation of 8 instances of the Merged By Playbook Instance task.

2. Because the Merged By Playbook Instance task is grouped by Playbook Instance, the merge occurs when the last execution path arrives at the task.

3. For each execution path, a process name is extracted with the JSON path $['Process Name'].returnData, specified by dataJsonPath in the Merged By Playbook Instance task.

4. All the extracted process names are grouped as an array and saved in the rawData field for the last instance of Merged By Playbook Instance task where the merge occurs.

5. When the merge occurs, the tasks linked from the Merged By Playbook Instance task are initiated. In this example, an instance of the Merged Result task, linked from the Merged By Playbook Instance task, is created.

Playbook Data

Playbook Data for last instance of Merged By Playbook Instance where the merge occurs:

Playbook Data for one instance of Merged By Playbook Instance where the merge is still ongoing:

Group by Task

Execution paths that contain the specified task name are grouped together and merged.

Sample 2: The following playbook demonstrates how to merge the execution paths by the task name Unwind2.

Condition

{
    "groupBy": "Task" ,
    "taskName" : "Unwind 2",
    "dataJsonPath": "$['Process Name'].returnData"
}

Description

1. There are 8 execution paths arriving at the Merged By Task Name task, resulting in the creation of 8 instances of the Merged By Task Name task.

2. Because the Merged By Task Name task is grouped by Task, the merge occurs when all the execution paths containing the specified task name, which is Unwind 2 in this example, arrive at the merge task.

3. In this example, the Unwind 2 task deconstructs an array containing two items and initiates two instances of the Extract Process Name 2 task. Therefore, there are two execution paths containing the task name Unwind 2 arriving at the merge task.

4. For each execution path, a process name is extracted with the JSON path $['Process Name'].returnData, specified by dataJsonPath in the Merged By Task Name task.

5. As a result, two extracted process names are grouped as an array and saved in the rawData field for the instance of Merged By Task Name task where the merge occurs.

6. When the merge occurs, the tasks linked from the Merge By Task Name task are initiated. In this example, an instance of the Merged Result task, linked from the Merged By Task Name task, is created.

Playbook Data

Playbook Data for the instance of Merged By Task Name where the merge occurs. This instance is the last instance of the merge task whose execution path contains the Unwind 2 task:

Playbook Data for one instance of Merged By Task Name where the merge is still ongoing:

Group by Task Instance

Execution paths that contain the same task instance number in the specified task are grouped together and merged.

Sample 3: The following playbook demonstrates how to merge the execution paths by the instance of the task Unwind1.

Condition

{
    "groupBy": "Task" ,
    "taskName" : "Unwind 1",
    "dataJsonPath": "$['Process Name'].returnData"
}

Description

1. There are 8 execution paths arriving at the Merged By Task Instance task, resulting in the creation of 8 instances of the Merged By Task Instance task.

2. The Merged By Task Instance task is grouped by Task Instance of the Unwind 1 task. The Unwind 1 task has two instances, each with a unique instance number. Both instances will pass through the Merged By Task Instance task. As as result, two merges will occur.

3. In this example, each instance of the Unwind 1 task deconstructs an array containing three items and initiates three instances of the Extract Process Name 1 task, resulting in a total of six instances of the Extract Process Name 1 task being created.

4. Although there are 8 execution paths arriving at the task Merged By Task Instance, two execution paths, which do not contain the Unwind 1task, are discarded. The other 6 execution paths, which contain the Unwind 1 task, are grouped into two groups, with each group of the execution paths containing the same instance number for the Unwind 1 task.

5. For each execution path being grouped, a process name is extracted with the JSON path $['Process Name'].returnData, specified by dataJsonPath in the Merged By Task Instance task

6. As a result, two groups of process names are created, with each group containing 3 process names. Each group of process names are saved as an array in the rawData field for the instance of Merged By Task Instance task where the merge occurs.

7. When the merge occurs, the tasks linked from the Merge By Task Instance task are initiated. In this example, two instances of the Merged Result task are created. Each is linked from one instance of the Merged By Task Instance task, where the merge occurs.

Playbook Data

Playbook Data for the instances of Merged By Task Instance where the merge occurs. Each instance is the last instance in one of two groups of the merge tasks whose execution path contains the same instance number of the Unwind 1 task:

Playbook Data for one instance of Merged By Task Instance where the merge is still ongoing:

Group by Path

Execution paths that contain the same value for the specified JSON paths are grouped together and merged.

Sample 4: The following playbook demonstrates how to merge the execution paths by specified JSON paths.

Condition

{
    "groupBy": "Path",
    "paths": ["$['Process Name'].returnData"]
}

Description

1. There are 8 execution paths arriving at the Merged By Path task, resulting in the creation of 8 instances of the Merged By Path task.

2. When each execution path arrives at the Merged by Path task, the execution path is grouped by the value, which is calculated by the JSON paths specified in paths of the condition.

3. In this example, 8 execution paths are grouped into 4 groups by the values wavessvc64.exe, winword.exe, excel.exe, wechat.exe.

4. Because the dataJsonPath is not specified in the condition, each group of execution paths are saved directly as an array in the rawData field for the instance of the Merge by Path task where the merge occurs.

5. When the merge occurs, the tasks linked from the Merge by Path task are initiated. In this example, four instances of the Merged Result task are created. Each is linked from one instance of the Merged by Path task, where the merge occurs.

**Playbook Data **

Three execution paths with process name winword.exe are grouped together and merged:

Two execution paths with process name excel.exe are grouped together and merged:

Two execution paths with process name wavesvc64.exe are grouped together and merged:

One execution path with process name wechat.exe are grouped together and merged:

Difference Between Merge and Wait for All

Wait for All can only merge execution paths from different tasks with only one execution path for one different task.

Sample 1

The following playbook demonstrates a simple scenario to use the Wait for All feature of the task to merge execution paths from two different tasks.

1. After the stage task Start is finished, one instance of the Test 1 task and one instance of the Test 2 task are initiated. As a result, one execution path started from the Start task is separated into two execution paths.

2. The instances of the Test 1 and Test 2 task are executed in parallel, so it is not certain which instance will be finished first.

3. Suppose the instance of the Test 1 task is finished first, an instance of the End task is initiated. However, the instance of the End task will not be executed immediately because its Run Mode is Wait for All, which means it has to wait for an instance of the Test 2 task to be finished.

4. After the instance of the Test 2 task is finished, the instance of the End task, which was created in step 3, starts executing.

5. After the instance of the End task is finished, the following execution path is generated:

   

If we use the Merge task to merge the execution paths for the above playbook, the playbook should look like:

1. After the stage task Start is finished, one instance of the Test 1 task and one instance of the Test 2 task are initiated. As a result, one execution path started from the Start task is separated into two execution paths.

2. The instances of the Test 1 and Test 2 task are executed in parallel, so it is not certain which instance will be finished first.

3. Suppose the instance of the Test 1 task is finished first, an instance of the Merge task is initiated and executed. However, the merge will not occur because it has to wait for the execution path from the instance of the Test 2 task.

4. After the instance of the Test 2 task is finished, another instance of the Merge task is initiated and executed. Because this is the last execution path arriving at the Merge task, the merge occurs and an instance of the End task is initiated.

5. After the instance of the task End is finished, the following execution path is generated:

   

In this example, both playbooks merge two execution paths into one but the execution path generated at the End task is different:

When execution paths are merged by Wait for All, all the task instances to be merged are part of the execution path for the task instance where the merge occurs. In this example, we can see the Test 1 and Test 2 task on the execution path of the End task.

When execution paths are merged by the Merge task, only the tasks on the execution path where the merge occurs are part of the execution path for the subsequent tasks. The merged data, an array containing two execution paths in this example, is saved in the rawData field in the Merge task. In this example, only the Test 2 task is part of the execution path of the End task. The Test 1 task is included in the rawData field of the Merge task.

Sample 2

Wait for All cannot merge the execution paths from the same task:

1. Two execution paths are generated after the instance of the Unwind task is completed,

2. Although the Run Mode for the End task is Wait for All, the two execution paths arriving at the End task are not merged because Wait for All can only merge one execution path from one task at a time. As a result, two instances of the End task are created.

In the following playbook, two execution paths are generated at the Test task following two other tasks, and the outcome is the identical:

If we use the Merge task to merge the execution paths in the same scenario, the execution paths can be merged correctly:

Sample 3

The following example demonstrates using Wait for All to merge execution paths from two different tasks, with one of the tasks having two instances. In this case, the execution path after merging has uncertainty.

1. There are three execution paths arriving at the End task. Two execution paths are from the Test 1 task, which has two instances, and one execution path is from the Test 2 task.

2. The Run Mode for the End task is configured as Wait for All, so the execution paths will be merged. However, only one of the execution paths from Test 1 task is merged with the execution path from the instance of the Test 2 task.

3. As a result, the task instance for the Test 1 task on the execution path of the End task could either be the first instance of the Test 1 task or the second instance of the Test 1 task, which is uncertain.

By using the Merge task in this scenario, the execution paths can be merged correctly:

As we can see in the execution path for the End task, three execution paths before the Merge task, with two from the Test 1 task and one from the Test 2 task, are merged as an array and saved in the rawData field of the task Merge:

Question

Can we use the following playbooks to merge execution paths from each instance of the Test task by applying Wait for All for the End task?

Both Test A and Test B task depend on the Test task. And on any one of two execution paths for the End task, the task instances for the Test A and Test B task must relate to the same instance of the Test task.

Playbook 1

Playbook 2