‎Dynamically Selecting Data

LAST UPDATED: FEB 14, 2025

Within a D3 playbook, data can be dynamically extracted across four difference sources:

Playbook Data Example

READER NOTE

Playbook data contains the cumulative set of information that flows through a playbook, up until the task previous to that of the current.

Suppose we have the following preprocessing playbook:

Click on Command Task 2, then open the format builder.
The listed items—DataSource, Trigger, Data Formatter Task 1, Conditional Task 1, and Data Formatter Task 3—represent elements that have been executed or triggered leading up to Command Task 2. Playbook tasks in parallel execution streams (i.e., Command Task 1 and Data Formatter Task 2) are excluded.

READER NOTE

Test running the playbook can aid in identifying and configuring the appropriate JSON path.

Suppose the relevant data being sought resides in the Return Data tab of Data Formatter Task 3.
Select the returnData field in the format builder, then click on the Save button.
Verify that {{ $.PlaybookData.["Data Formatter Task 3"].returnData }} has rendered, then click on the button.

UNDERSTANDING THE EXPRESSION

{{ ... }}: The double curly braces signify that the enclosed content is a dynamic expression that will be processed using Jinja and replaced with its resolved value.
- $.PlaybookData: This refers to the entire data structure associated with Playbook Data, starting from the root level.
["Data Formatter Task 3"]: The square brackets are used to access a specific task node that appears after the trigger, in this case, "Data Formatter Task 3."
.returnData: This retrieves the return data output from "Data Formatter Task 3."

Click on the icon of Command Task 2 after (re)testing the playbook to verify data extraction.

Local Shared Data Example

Configure an auto-run Add Root Key for Local Shared Data utility command task.

INPUT

JSON

{
    "url": "https://docs.d3security.com/user-docs/16.8/d3-glossary",
    "glossary": {
        "A": {
            "Ad-hoc Command": "An integration command or utility command that can be executed outside of a playbook.",
            "Ad-hoc Task": "A manually created incident response task within an incident workspace, requiring a task name and assignee, with optional fields for task type, description, due date, and mandatory completion before closure.",
            "Artifact": "A piece of evidence or data extracted from external systems that can be linked to D3 events and incidents to support analysis and automation.",
            "Automation Rule": "A user-defined set of criteria and actions used to automatically evaluate, categorize, escalate, dismiss, ignore, or link events and incidents within D3."
        },
        "B": {},
        "C": {
            "Codeless Playbook": "A method for implementing utility and integration commands in the form of a playbook.",
            "Command": "A directive that can execute actions in automation workflows or as standalone tasks.",
            "Command Task": "A type of playbook node that executes a specific integration or utility command.",
            "Configuration": "The set of properties, parameters, rules, or keys governing how one or more components operate within the platform.",
            "Connection": "A link that enables communication and interaction between D3 and an integration.",
            "Correlation (Event-Incident)": "The process of evaluating ingested activity data against escalation criteria to determine its significance and later, if the criteria are met, linking an event to an incident.",
            "Custom View": "A tailored view of data within the Investigation Dashboard, presenting a table with uniquely selected attribute columns, groupings, and sorting criteria to organize information and enable quick navigation to the desired details."
        },
        "D": {
            "D3 Forms": "A web-based tool for incident reporting and escalation of suspicious activities.",
            "Data Formatter Task": "A type of playbook task for creating, selecting, and manipulating data.",
            "Dynamic Form": "See Incident Form."
        },
        "E": {
            "Enrichment": "The process of augmenting raw data with additional context to enhance its value and relevance for analysis.",
            "Escalation": "The initial association of an event with an incident after it is identified as a potential security threat requiring further investigation and response."
        },
        "F": {
            "Fetch Event": "A command that retrieves data from an integration and encapsulates it into a D3 event.",
            "Fetch Incident": "A command that retrieves data from an integration and encapsulates it into a D3 incident."
        },
        "G": {
            "Global List": "A centralized repository for storing reusable JSON data sets to support dynamic operations."
        },
        "H": {},
        "I": {
            "Incident": "A unit of investigation—created either manually through D3 Forms or the vSOC navigation bar, or automatically via a preprocessing playbook—containing information necessary to facilitate response and remediation.",
            "Incident Field Mapping": "The process of mapping data fields within fetched JSON data to user-defined and D3 data model fields, through the use of the Fetch Incident command."
        },
        "J": {
            "Jinja": "A Python-syntax-inspired templating language used within D3 playbooks to dynamically retrieve, manipulate, and present data.",
            "JSON Path": "A syntax used to traverse and query elements within a JavaScript Object Notation (JSON) document. It defines the path to a particular value within a nested JSON structure, allowing users to extract or reference specific data based on its hierarchy."
        },
        "K": {},
        "L": {
            "Linking (Event / Incident)": "See Correlation.",
            "Local Shared Data": "JSON data specific to a single playbook, shared among all tasks within that playbook."
        },
        "M": {
            "Merge Task": "A type of playbook task that helps consolidate data from selected playbook execution paths into a single array, aiding data management and processing."
        },
        "N": {},
        "O": {
            "Organization Management": "A module that enables administrators to configure users, groups, roles, and sites to manage access control within D3."
        },
        "P": {
            "Passdown Task": "A type of playbook task that processes or prepares return data from the playbook for use in other destinations.",
            "Pending Task": "Any playbook task that is in a pending state, thereby preventing the execution from advancing to subsequent playbook tasks.",
            "Playbook": "A predefined automation workflow consisting of task nodes working together to perform security operations."
        },
        "Q": {},
        "R": {
            "Raw Data": "The unprocessed and unformatted information ingested from integrations.",
            "RBAC": "A security model that allows the enforcement of access policies by defining roles and assigning them to users."
        },
        "S": {
            "Schedule": "A configuration that automates the periodic execution of integration and utility commands.",
            "Severity": "The level of impact or criticality assigned to an incident, used to prioritize response efforts."
        },
        "T": {
            "Task": "The building blocks of a playbook, linked via a control flow, performing specific actions or functions.",
            "Task Status": "The current state of a playbook task during testing or live execution."
        }
    }
}

JSON PATH

CODE

$.Demo

Test run the playbook, then click on the icon to verify data entry.
Connect a data formatter task, then select the Local Shared Data’s definition.

UNDERSTANDING THE EXPRESSION

{{ ... }}: The double curly braces signify that the enclosed content is a dynamic expression that will be processed using Jinja and replaced with its resolved value.
SharedContextData: The operand holding the JSON object for this playbook’s locally shared data. It is used as input for the filter applied on the right side of the pipe (|).
jsonpath('$.Demo.glossary.L["Local Shared Data"]'): The filter applied to SharedContextData
- $.Demo.glossary.L["Local Shared Data"] navigates through the JSON structure configured in step 1:
  - $.Demo refers to the root object named Demo.
  - .glossary accesses the glossary JSON key within Demo object.
  - .L["Local Shared Data"] retrieves the definition of "Local Shared Data" within the "L" section of the glossary.

Run the data formatter task, then navigate its Return Data tab to verify data extraction.

Global List Example

Suppose we are looking for all objects within a "Suspicious IPs" global list:

Ensure permission is granted for the playbook test run site (in step 3b).
Configure an auto-run data formatter task, selecting from the Suspicious IPs global list.
Test run the playbook.
Navigate to the data formatter task’s Return Data tab to verify data extraction.

Trigger Output Data Example

READER NOTE

Trigger output data (TOD) is the JSON—initially generated behind the scenes during field mapping and refined in the On Event Ingestion workflow—that contains all information required to create and escalate or dismiss a D3 event.

Suppose we want to modify the Dismiss field of TOD from false to to true.

Set up an auto-run Update Root Key Value for Trigger Output Data with JSON Path utility command task, deactivate it, then test the playbook.
Verify that the TOD’s Dismiss field is false
Reactivate the task, then configure it.
SOURCE TYPE
Trigger Output Data
JSON PATH
$.Dismiss
UPDATE VALUE
true
Click on the Stop Test button, then re-test the playbook.
Navigate to the Context Data tab of the task to verify the update.
Alternatively, click on the icon.