Skip to main content
Skip table of contents

Field Mappings

LAST UPDATED: JULY 28, 2025

app.diagrams.net_ 1-20250725-002356.png

Third-party vendors provide semantically distinct entities—such as events, incidents, and alerts—using varied JSON structures and key names. To normalize inter-vendor and payload heterogeneity for streamlined processing and operational support, raw JSON keys containing relevant event- or incident-related information are reassigned to D3 data model or user-defined field names. This key reassignment process is known as field mapping.

IFM and EFM in the Data Pipeline

Field Mapping Types

There are two field mapping types: Incident Field Mapping (IFM) for incident-level data and Event Field Mapping (EFM) for event-level data. Both occur in the post-ingestion processing pipeline, which varies slightly based on the integration fetch command used.

46523396-3a06-45ef-b185-77a417bf1961.png

Fetch commands are of two main types:

  • Fetch Incident – Applies both incident field mappings and event field mappings.

  • Fetch Event – Applies only event field mappings.

Fetch Incident - Post-Ingestion Process Flow (IFM and EFM)

FLOW DIAGRAM

Frame 4 (1)-20250722-184013.png

The array data representing a D3 event is nested within the array data representing a D3 incident.

INCIDENT-EVENT PAYLOAD

image 36-20250724-190926.png
Fetch Event - Post-Ingestion Process Flow (EFM)

FLOW DIAGRAM

Frame 5 (1)-20250722-184033.png

EVENT PAYLOAD

image 35-20250724-191000.png

Source Field as Individual Mappings

A source field is a relative JSONPath expression used in EFM and IFM to reference a specific field within the raw JSON payload. The source field begins at the array defined by the Main JSON Path (IFM) or Main Event JSON Path (EFM), and traverses deeper into the JSON structure to reach the desired key.

image 55-20250728-164431.png

See Source Field — Supported JSONPath Expressions for syntax details.

Source Resolution

Each event or incident source defines how data for a vendor-specific entity type should be interpreted and normalized. In D3, source is implemented as an integration-specific, named configuration that links to an EFM or IFM set. To apply the correct set, D3 must resolve each object within the array defined by the Main (Event) JSON Path to a specific source.

The Main Event JSON Path points to an array, with each object evaluated against configured EFM sources. The Main JSON Path also points to an array, with each object evaluated against configured IFM sources and potentially containing a nested array referenced by the Main Event JSON Path. A source's applicability is determined by a search string, which may be built-in or user-defined.

Example - Event Sources and Search Strings (Zendesk)

OBJECTIVE – Understand the relationship between event sources and search strings.

The Zendesk integration includes five built-in event sources:

  • (Default Event Source)

  • Event Source for Ticket

  • Event Source for User

  • Event Source for Group

  • Event Source for Organization

Event Source for Organization *

PARTIAL PAYLOAD 1

JSON 
{
    "results": [
        {
            ...
            "result_type": "organization",
            ...
        }
    ]
}

  1. Select the Event Source for Organization.

    Frame 1-20250716-015114.png

  2. Click on the Edit Event Source button.

    Frame 6-20250716-174418.png

  3. Observe the alignment between the search string and the payload's source discriminator.

    Frame 8-20250716-180145.png

Event Source for Group *

PARTIAL PAYLOAD 2

JSON 
{
    "results": [
        {
            ...
            "result_type": "group",
            ...
        }
    ]
}

  1. Select the Event Source for Group.

    Frame 2 (4)-20250716-164738.png

  2. Click on the Edit Event Source button.

    Frame 6-20250716-174418.png

  3. Observe the alignment between the search string and the payload's source discriminator.

    Frame 9-20250716-180216.png

Event Source for User *

PARTIAL PAYLOAD 3

JSON 
{
    "results": [
        {
            ...
            "result_type": "user",
            ...
        }
    ]
}

  1. Select the Event Source for User.

    Frame 3 (1)-20250716-171533.png

  2. Click on the Edit Event Source button.

    Frame 6-20250716-174418.png

  3. Observe the alignment between the search string and the payload's source discriminator.

    Frame 10-20250716-180239.png

Event Source for Ticket *

PARTIAL PAYLOAD 4

JSON 
{
    "results": [
        {
            ...
            "result_type": "ticket",
            ...
        }
    ]
}

  1. Select the Event Source for Ticket.

    Frame 4-20250716-171622.png

  2. Click on the Edit Event Source button.

    Frame 6-20250716-174418.png

  3. Observe the alignment between the search string and the payload's source discriminator.

    Frame 12-20250716-180302.png

(Default Event Source) *

PARTIAL PAYLOAD 5

JSON 
{
    "results": [
        {
            ...
            "result_type": "neither_ticket_user_group_nor_organization",
            ...
        }
    ]
}

  1. Select the (Default Event Source).

    Frame 5-20250716-171705.png

  2. Click on the Edit Event Source button.

    Frame 6-20250716-174418.png

  3. Observe the Main Even JSON Path.

    Frame 11-20250716-180326.png

NOTES ON DEFAULT EVENT SOURCE *

  • The Event Source popover for (Default Event Source) includes a Main Event JSON Path field instead of a Search String field.

  • When no search string matches the value of the target field in the payload, the D3 system evaluates the Main Event JSON Path.

    • If the path resolves successfully, the default field mapping set is applied.

    • If the path fails to resolve, no field mapping is applied.

Example - Incident Sources and Search Strings (Microsoft Sentinel)

OBJECTIVE – Understand the relationship between incident sources and search strings.

PARTIAL PAYLOAD 1

JSON 
{
    "value": [
        {
            ...
            "discriminator": "lorem_ipsum",
            "properties": {
                ...
                "title": "demo_title",
                "description": "demo_description",
                "incidentNumber": 133,
                ...
            },
            ...
            "alertDetails": [
                {
                    ...
                    "Tactics": "CredentialAccess",
                    "Techniques": "",
                    "Type": "SecurityAlert"
                    ...
                }
            ]
            ...
        }
    ]
}

  1. Add an incident source, within a site, using the following search string:

    CODE 
    {.discriminator}={{ipsum$}}
    Frame 106-20250723-181307.png

  2. Observe the alignment between the search string and the payload's source discriminator.

PARTIAL PAYLOAD 2

JSON 
[
    {
        ...
        "discriminator": "value_for_demo_incident_source",
        ...
    }
]

  1. Add an incident source, within a site, using the following search string:

    CODE 
    {$..discriminator}=value_for_demo_incident_source
    Frame 103 (1)-20250722-205814.png

  2. Observe the alignment between the search string and the payload's source discriminator.

Search Strings

Each non-default source involves a search string that determines its applicability during source resolution. A search string evaluates data within the Main (Event) JSON Path to determine whether a specific key exists and whether its value matches the configured value. When a key-value pair in the payload matches a configured search string, the corresponding EFM or IFM set is used to normalize the third-party data.

The search string must adhere to one of the following formats:

FORMAT 1

CODE 
{<Descriminator JSONPath>}=<Discriminator's literal value without quotes>

FORMAT 2

CODE 
{<Descriminator JSONPath>}={{<Regular expression matching discriminator's value>}}
Testing EFM – Search Strings, Source Field, Data Ingestion (Zendesk)

TEST 1

  1. Navigate to the Fetch Event command of the Zendesk integration.

    Frame 42 (5)-20250630-175639.png

  2. Enable webhook authentication, then select the API Key option.

    Frame 39 (1)-20250630-183146.png

  3. Create an API key. Note the selected site.

    Frame 47 (2)-20250630-185629.png

  4. Copy the request information to Postman, then select a preprocessing playbook under the Additional Settings accordion.

    Frame 46 (2)-20250630-185035.png

    In Postman:

    • Prepare the POST request URL.

      Frame 21 (1)-20250718-190151.png

    • Prepare the request header key and value.

      Frame 22 (1)-20250718-190210.png

  5. Prepare the following payload for ingestion:

    JSON 
    {
    "results": [
        {
            "id": 1,
            "discriminator": "value_for_demo_event_source_1",
            "layer1": {
                "layer2": {
                    "layer3": {
                       "target": "demo_value_1"
                    }
                }
            }
        }
    ]
}
    Frame 23 (1)-20250718-190546.png

  6. Click on the Set Up Event Field Extraction Mapping button in D3 vSOC.

    Frame 41-20250630-175547.png

  7. Note the Main Event JSONPath expression.

    Frame 2 (4)-20250630-175827.png

WHAT IT MEANS *

  • At runtime, the $.results expression directs the system to iterate over each object in the top-level "results" array.

    • If this array is not at the top level, the Main Event JSON Path must include either the full path of parent objects or (cautiously) use recursive descent to locate it.

    • The source field (step 8) operates relative to each object in this scope.

  1. Create an event source called Demo Event Source 1 using the following search string:

    CODE 
    {$.discriminator}=value_for_demo_event_source_1

    This instructs the system to evaluate the discriminator key in the payload, and apply this event source when its value equals the string "value_for_demo_event_source_1".

    Frame 19 (1)-20250718-215452.png

  2. Add a mapping field for Demo Event Source 1:

    Frame 24-20250718-214924.png

RELATED RESOURCE *

Learn more about JSONPath expressions for source field here.

  1. Send the POST request prepared in step 4 and step 5.

    Frame 26-20250718-222419.png

  2. Verify that an event was created in the Data Ingestion module.

    Frame 81 (1)-20250725-165556.png

  3. Note the eventId number.

    Frame 82 (1)-20250725-165621.png

  4. Navigate to the Investigation Dashboard page.

    Frame 49 (1)-20250630-190451.png

  5. Select the site in which the API key was generated in step 3.

    Frame 9 (1)-20250630-190521.png

  6. Locate the event with an ID matching the one observed in step 12, then click on the event to view its details.

    Frame 163-20250725-165825.png

  7. Verify that the value from the "target" payload field appears next to the mapped field (Demo Field 1 configured in step 9).

    Frame 84 (1)-20250725-165918.png

TEST 2

  1. Prepare the following payload for ingestion:

    JSON 
    {
    "results": [
        {
            "id": 2,
            "discriminator": "value_for_demo_event_source_2",
            "layer1": {
                "layer2": {
                    "layer3": {
                       "target": "demo_value_2"
                    }
                }
            }
        }
    ]
}
    Frame 86-20250718-233539.png

  2. Create an event source called Demo Event Source 2 using the following search string:

    CODE 
    {$.discriminator}={{2$}}​

    This instructs the system to evaluate the discriminator key in the payload, and apply this event source when its value matches any string that ends with "2".

    Frame 20-20250718-215718.png

  3. Add a mapping field for Demo Event Source 2:

    image-20250718-234229.png

  4. Send the POST request.

    image-20250718-234448.png

  5. Verify that an event was created in the Data Ingestion module.

    Frame 93-20250719-001150.png

  6. Note the eventId number.

    Frame 94-20250719-001051.png

  7. Navigate to the Investigation Dashboard page.

    Frame 49 (1)-20250630-190451.png

  8. Select the site in which the API key was generated in step 3.

    Frame 9 (1)-20250630-190521.png

  9. Locate the event with an ID matching the one observed in step 22, then click on the event to view its details.

    Frame 95-20250719-001731.png

  10. Verify that the value from the "target" payload field appears next to the mapped field (Demo Field 2) configured in step 19.

    Frame 96-20250719-001958.png
Testing IFM & EFM – Search Strings, Source Field, Data Ingestion (Microsoft Sentinel)

PART 1

  1. Navigate to the Fetch Incident command of the Microsoft Sentinel integration.

    Frame 107-20250723-183050.png

  2. Enable webhook authentication, then select the API Key option.

    Frame 105-20250723-183131.png

  3. Create an API key. Note the selected site.

    Frame 109 (1)-20250723-184346.png

  4. Copy the request information to Postman, then select a preprocessing playbook under the Additional Settings accordion.

    Frame 108 (1)-20250723-183456.png

    In Postman:

    • Prepare the POST request URL.

      Frame 114-20250723-190237.png

    • Prepare the request header key and value.

      Frame 115-20250723-190338.png

  5. Prepare the following payload for ingestion:

    JSON 
    {
    "value": [
        {
            "discriminator": "lorem_ipsum",
            "properties": {
                "title": "demo_title",
                "description": "demo_description",
                "incidentNumber": 99999
            },
            "alertDetails": [
                {
                    "Tactics": "CredentialAccess",
                    "Techniques": "",
                    "Type": "demo_type",
                    "SystemAlertId": "3f8a1b94-b79e-4b65-beb7-2a14f0a6e5a9",
                    "Placeholder": "demo_value"
                }
            ]
        }
    ]
}
    Frame 125-20250723-220214.png

  6. Click on the Set Up Incident Field Mapping button in D3 vSOC.

    Frame 113-20250723-185802.png

  7. Add an incident source, within a site, using the following search string:

    CODE 
    {.discriminator}={{ipsum$}}
    Frame 106 (1)-20250723-191208.png

  8. Click on and configure the following system fields.

    Frame 131-20250723-230320.png

  9. Verify that the IFM reflects the updated source fields.

    Frame 133-20250723-231132.png

  10. Click on the Set Up Event Field Extraction Mapping button.

    Frame 119-20250723-193327.png

  11. Set the Main Event JSONPath to $.alertDetails.

    Frame 120-20250723-194142.png

  12. Create an event source called Demo Event Source using the following search string:

    CODE 
    {$.Type}=demo_type
    Frame 124-20250723-214403.png

    This instructs the system to evaluate the $.value.alertDetails.Type key in the payload, and apply this event source when its value equals the string "demo_type".

  13. Click on the + Add Field button to configure the following fields:

    Frame 132-20250723-230742.png

PROGRESS SUMMARY

  • The value array serves as the entry point for the IFM.

    • Each unique object within it (based on incidentNumber) contains data used to generate a D3 incident.

    • Three IFMs are configured.

  • The alertDetails array serves as the entry point for the EFM.

    • Each unique object within it (based on SystemAlertId) contains data used to generate a D3 event.

    • Four EFMs are configured.

      • A user-defined Demo Key mapping field is set to extract D3 event data from the path $.value.alertDetails.Placeholder.

  1. Send the POST request prepared in step 5.

    Frame 127-20250723-221831.png

  2. Verify that a D3 incident was created in the Data Ingestion module, noting the IncidentNo value.

    Frame 134-20250723-232956.png

  3. Navigate to the Investigation Dashboard page.

    Frame 49 (1)-20250630-190451.png

  4. Select the site in which the API key was generated in step 3.

    Frame 9 (1)-20250630-190521.png

  5. Locate the incident by its unique incident number from step 15, then select it to view details.

    Frame 135-20250723-234038.png

  6. Verify that the incident workspace reflects the two other values configured in step 9.

    Frame 138-20250724-004718.png

  7. Click on the Events sidebar navigation icon, then click on the event row entry.

    Frame 136-20250723-235907.png

  8. Verify that the four values configured in step 13 are displayed.

    Frame 137-20250724-002741.png

Order Priority

When search strings from different sources simultaneously match different key-value pairs in the ingested JSON payload, the Order Priority field determines which match takes precedence.

Frame 139-20250724-175028.png

  • The (Default Event Source) mappings apply regardless of Order Priority.

  • Only one non-default event source applies per post-ingestion pipeline execution.

  • Search strings from multiple event sources may match fields within the same payload.

    • The source with the lower Order Priority takes precedence.

    • If priorities are equal, the source created first takes precedence.

Testing Order Priority (Zendesk)

OBJECTIVE – Understand the effect of order priority when search strings from different sources simultaneously match different key-value pairs in the ingested JSON payload.

  1. Create the following two event sources, each configured with the default Order Priority value.

    Frame 161-20250724-235302.png

    SEARCH STRING 1

    CODE 
    {$.discriminator1}=value1

    SEARCH STRING 2

    CODE 
    {$.discriminator2}=value2

  2. Prepare the following payload for ingestion:

    Frame 145-20250724-214641.png
    JSON 
    {
    "results": [
        {
            "id": "902",
            "discriminator1": "value1",
            "discriminator2": "value2",
            "target0": "(Default Event Source) matched",
            "target1": "Demo Event Source 1 matched",
            "target2": "Demo Event Source 2 matched"
        }
    ]
}

  3. Add the following field mappings:

    1. (Default Event Source):

      Frame 141-20250724-212651.png

    2. Demo Event Source 1:

      Frame 142-20250724-212720.png

    3. Demo Event Source 2:

      Frame 143-20250724-212732.png

  4. Send the POST request.

    Frame 146-20250724-214836.png

  5. Verify that an event was created in the Data Ingestion module.

    Frame 147-20250724-220041.png

  6. Note the eventId number.

    Frame 150-20250724-221710.png

  7. Navigate to the Investigation Dashboard page.

    Frame 49 (1)-20250630-190451.png

  8. Select the site in which the API key was generated.

    Frame 9 (1)-20250630-190521.png

  9. Locate the event with an ID matching the one observed in step 6, then click on the event to view its details.

    Frame 149-20250724-221539.png

  10. Verify that the Unique Event Key matches the id specified in step 2, and that both the mappings for the (Default Event Source) and Demo Event Source 1 are displayed.

    Frame 160-20250724-233308.png

    Demo Event Source 1 matched because it was created first in step 3.

  11. Modify the Order Priority of Demo Event Source 1 to a value that is lower than that of Demo Event Source 2.

    Frame 152-20250724-222314.png

  12. Change the id field in the payload in Postman to a new, previously unused value.

    Frame 155 (1)-20250724-223443.png

  13. Repeat steps 4-9 for the newly ingested data.

  14. Verify that the Unique Event Key matches the id specified in step 12, and that, again, both the mappings for the (Default Event Source) and Demo Event Source 1 are displayed.

    Frame 156-20250724-223950.png

  15. Modify the Order Priority of Demo Event Source 2 to a value that is lower than that of Demo Event Source 1.

    Frame 157-20250724-224222.png

  16. Change the id field in the payload in Postman to a new, previously unused value.

    Frame 158-20250724-224648.png

  17. Repeat steps 4-9 for the newly ingested data.

  18. Verify that the Unique Event Key matches the id specified in step 16, and that both the mappings for the (Default Event Source) and Demo Event Source 2 are displayed.

    Frame 159-20250724-224806.png

RELATED ARTICLES

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close