Field Mappings

last updated: November 13, 2026

app.diagrams.net_ 1-20251021-201840.png

Third-party vendors provide semantically distinct entities—such as events, incidents, and alerts—using varied JSON structures and key names. To normalize inter-vendor and payload heterogeneity for streamlined processing and operational support, raw JSON keys containing relevant event- or incident-related information are reassigned to D3 data model or user-defined field names. This key reassignment process is known as field mapping.

IFM and EFM in the Data Pipeline

Field Mapping Types

There are two field mapping types: Incident Field Mapping (IFM) for incident-level data and Event Field Mapping (EFM) for event-level data. Both occur in the post-ingestion processing pipeline, which varies slightly based on the integration fetch command used.

46523396-3a06-45ef-b185-77a417bf1961.png


Fetch commands are of two main types:

  • Fetch Incident – Applies both incident field mappings and event field mappings.

  • Fetch Event – Applies only event field mappings.

Fetch Incident - Post-Ingestion Process Flow (IFM and EFM)

flow diagram

Frame 4 (1)-20250722-184013.png

The array data representing a D3 event is nested within the array data representing a D3 incident.

incident-event payload

image 36-20250724-190926.png
Fetch Event - Post-Ingestion Process Flow (EFM)

flow diagram

Frame 5 (1)-20250722-184033.png

event payload

image 35-20250724-191000.png

Source Field as Individual Mappings

A source field is a relative JSONPath expression used in EFM and IFM to reference a specific field within the raw JSON payload. The source field begins at the array defined by the Main JSON Path (IFM) or Event Root Path (EFM), and traverses deeper into the JSON structure to reach the desired key.

fm186-concept-key-path.png

See Source Field — Supported JSONPath Expressions for syntax details.

WHEN THE RAW DATA DOES NOT FIT

D3’s system integrations already define the default Main JSON Path, Event Root Path and field mappings. If a data source sends its raw data in a different structure, but the data should still fit those paths and field mappings, you do not need to rebuild the mappings. Use an event formatter to reshape the raw data so that it fits the existing paths and field mappings before D3 processes it. Event formatters are available on webhook data sources and scheduled fetches; on a webhook data source, choose it under Additional Settings > Data Formatter. See Event Formatter.

Source Resolution

Each event or incident source defines how data for a vendor-specific entity type should be interpreted and normalized. In D3, source is implemented as an integration-specific, named configuration that links to an EFM or IFM set. To apply the correct set, D3 must resolve each object within the array defined by the Main JSON Path or Event Root Path to a specific source.

The Event Root Path points to an array, with each object evaluated against configured EFM sources. The Main JSON Path also points to an array, with each object evaluated against configured IFM sources and potentially containing a nested array referenced by the Event Root Path. A source's applicability is determined by a search string, which may be built-in or user-defined.

Example - Event Sources and Search Strings (Zendesk)

objective – Understand the relationship between event sources and search strings.


The Zendesk integration includes five built-in event sources. Each is a mapping set, listed under Field Mapping Sets in the Event Field Mapping window of the Fetch Event command:

  • (Default Event Source)

  • Event Source for Ticket

  • Event Source for User

  • Event Source for Group

  • Event Source for Organization

Event Source for Organization *

partial payload 1

JSON
{
    "results": [
        {
            ...
            "result_type": "organization",
            ...
        }
    ]
}
  1. Under Field Mapping Sets, select Event Source for Organization.

    Event Field Mapping window with Event Source for Organization selected under Field Mapping Sets
  2. Select the edit (pencil) icon on the mapping set.

    The edit (pencil) icon on the Event Source for Organization mapping set
  3. Observe the alignment between the mapping set's Condition and the payload's source discriminator.

    fm186-zen-condition-organization.png

Event Source for Group *

partial payload 2

JSON
{
    "results": [
        {
            ...
            "result_type": "group",
            ...
        }
    ]
}
  1. Under Field Mapping Sets, select Event Source for Group.

    Event Field Mapping window with Event Source for Group selected under Field Mapping Sets
  2. Select the edit (pencil) icon on the mapping set.

    The edit (pencil) icon on the Event Source for Group mapping set
  3. Observe the alignment between the mapping set's Condition and the payload's source discriminator.

    fm186-zen-condition-group.png

Event Source for User *

partial payload 3

JSON
{
    "results": [
        {
            ...
            "result_type": "user",
            ...
        }
    ]
}
  1. Under Field Mapping Sets, select Event Source for User.

    Event Field Mapping window with Event Source for User selected under Field Mapping Sets
  2. Select the edit (pencil) icon on the mapping set.

    The edit (pencil) icon on the Event Source for User mapping set
  3. Observe the alignment between the mapping set's Condition and the payload's source discriminator.

    fm186-zen-condition-user.png

Event Source for Ticket *

partial payload 4

JSON
{
    "results": [
        {
            ...
            "result_type": "ticket",
            ...
        }
    ]
}
  1. Under Field Mapping Sets, select Event Source for Ticket.

    Event Field Mapping window with Event Source for Ticket selected under Field Mapping Sets
  2. Select the edit (pencil) icon on the mapping set.

    The edit (pencil) icon on the Event Source for Ticket mapping set
  3. Observe the alignment between the mapping set's Condition and the payload's source discriminator.

    fm186-zen-condition-ticket.png

(Default Event Source) *

partial payload 5

JSON
{
    "results": [
        {
            ...
            "result_type": "neither_ticket_user_group_nor_organization",
            ...
        }
    ]
}
  1. Under Field Mapping Sets, select (Default Event Source).

    Event Field Mapping window with (Default Event Source) selected
  2. Note that (Default Event Source) has no condition. Its scope is set by the Event Root Path at the top of the window; select Edit to change it.

    The Edit button next to Event Root Path
  3. Observe the Event Root Path.

    fm186-zen-root-path.png

NOTES ON DEFAULT EVENT SOURCE *

  • (Default Event Source) has no condition. Its scope is the Event Root Path at the top of the Event Field Mapping window.

  • When no search string matches the value of the target field in the payload, the D3 system evaluates the Event Root Path.

    • If the path resolves successfully, the default field mapping set is applied.

    • If the path fails to resolve, no field mapping is applied.

Example - Incident Sources and Search Strings (Microsoft Sentinel)

objective – Understand the relationship between incident sources and search strings.


partial payload 1

JSON
{
    "value": [
        {
            ...
            "discriminator": "lorem_ipsum",
            "properties": {
                ...
                "title": "demo_title",
                "description": "demo_description",
                "incidentNumber": 133,
                ...
            },
            ...
            "alertDetails": [
                {
                    ...
                    "Tactics": "CredentialAccess",
                    "Techniques": "",
                    "Type": "SecurityAlert"
                    ...
                }
            ]
            ...
        }
    ]
}
  1. Add an incident source, within a site, using the following search string:

    {.discriminator}={{ipsum$}}
    
    Add Source panel with a regex search string
  2. Observe the alignment between the search string and the payload's source discriminator.


partial payload 2

JSON
[
    {
        ...
        "discriminator": "value_for_demo_incident_source",
        ...
    }
]
  1. Add an incident source, within a site, using the following search string:

    {$..discriminator}=value_for_demo_incident_source
    
    Add Source panel with a literal search string
  2. Observe the alignment between the search string and the payload's source discriminator.

Search Strings

Each non-default source involves a search string that determines its applicability during source resolution. A search string evaluates data within the Main JSON Path or Event Root Path to determine whether a specific key exists and whether its value matches the configured value. When a key-value pair in the payload matches a configured search string, the corresponding EFM or IFM set is used to normalize the third-party data.

The search string must adhere to one of the following formats:

format 1

{<Descriminator JSONPath>}=<Discriminator's literal value without quotes>

format 2

{<Descriminator JSONPath>}={{<Regular expression matching discriminator's value>}}
Testing EFM – Search Strings, Source Field, Data Ingestion (Zendesk)

test 1

  1. Navigate to the Fetch Event command of the Zendesk integration.

    Zendesk commands list with Fetch Event
  2. Under External Access, turn on Accept events from outside D3, then select Webhook keys.

    fm186-zen-external-access.png
  3. Go to Data Ingestion > Event Intake, select New > Webhook, then choose the Zendesk integration and a site. Under Webhook Keys, select Generate a Key, enter a key name, and select Generate. Note the selected site.

    fm186-k-ingest-new-menu.png
    fm186-k-key-generate.png
  4. Copy the Request URL, the Request Header Key (d3key) and the Request Header Value to Postman. Optionally, select a preprocessing playbook under Additional Settings.

    Request URL, request header key d3key and the masked header value; Additional Settings with Data Formatter and Preprocessing Playbook

    In Postman:

    • Prepare the POST request URL.

      Frame 21 (1)-20250718-190151.png
    • Prepare the request header key and value.

      Frame 22 (1)-20250718-190210.png
  5. Prepare the following payload for ingestion:

    JSON
    {
        "results": [
            {
                "id": 1,
                "discriminator": "value_for_demo_event_source_1",
                "layer1": {
                    "layer2": {
                        "layer3": {
                           "target": "demo_value_1"
                        }
                    }
                }
            }
        ]
    }
    

    Frame 23 (1)-20250718-190546.png
  6. Under Event Field Mapping, select Setup Event Field Mapping.

    The Setup Event Field Mapping button
  7. Note the Event Root Path.

    fm186-zen-root-path.png

WHAT IT MEANS *

  • At runtime, the $.results expression directs the system to iterate over each object in the top-level "results" array.

    • If this array is not at the top level, the Event Root Path must include either the full path of parent objects or (cautiously) use recursive descent to locate it.

    • The source field (step 8) operates relative to each object in this scope.

  1. Add a mapping set called Field Mappings Demo Event Source 1. Open the Field Mapping Sets menu (…) and select Add Mapping Set, then enter the name. Under Condition, select … to open the Source Field JSON Picker. The picker lists the fields of a sample payload: select + and add the payload from step 5 as custom sample data, then select discriminator and Save. Enter value_for_demo_event_source_1 as the value and select the check mark. The set’s condition reads:

    {$.discriminator}=value_for_demo_event_source_1
    

    This instructs the system to evaluate the discriminator key in the payload, and apply this event source when its value equals the string "value_for_demo_event_source_1".

    fm186-k-sets-menu-add.png
    Custom Sample Data with the demo payload
    fm186-k-picker-condition.png
    fm186-k-new-set-1.png
  2. Select Field Mappings Demo Event Source 1, then select Add Mapping. Under Source fields from, select … and pick layer1 > layer2 > layer3 > target. Under Destination fields in D3, search for Description and select it, then select Save.

    fm186-k-picker-source.png
    fm186-k-dest.png
    fm186-k-add-mapping-1.png
    Field Mappings Demo Event Source 1 with its mapping


RELATED RESOURCE *

Learn more about JSONPath expressions for source field here.

  1. Send the POST request prepared in step 4 and step 5.

    Frame 26-20250718-222419.png
  2. Verify that an event was created: in the sidebar, go to Events, open the view menu and select All Events.

    fm186-k-events-views.png
  3. Note the EventID of the new event. Its Data Source is Zendesk and its Event Source is the mapping set that matched, Field Mappings Demo Event Source 1.

    fm186-k-events-list-1.png
  4. If the list shows more than one site, select the site from step 3 in the site selector.

  5. Select the EventID from step 12 to open the event’s details.

  6. Verify that the Unique Event Key is the id from the payload in step 5 (1).

    fm186-k-event1-key.png
  7. Verify that the value from the "target" payload field appears in Description, the destination field configured in step 9).

    fm186-k-event1-desc.png

test 2

  1. Prepare the following payload for ingestion:

    JSON
    {
        "results": [
            {
                "id": 2,
                "discriminator": "value_for_demo_event_source_2",
                "layer1": {
                    "layer2": {
                        "layer3": {
                           "target": "demo_value_2"
                        }
                    }
                }
            }
        ]
    }
    

    Frame 86-20250718-233539.png


  2. Add a mapping set called Field Mappings Demo Event Source 2 as in step 8: pick discriminator in the Source Field JSON Picker and enter {{2$}} as the value. The set’s condition reads:

    {$.discriminator}={{2$}}
    

    This instructs the system to evaluate the discriminator key in the payload, and apply this event source when its value matches any string that ends with "2".

    fm186-k-new-set-2.png
  3. Add a mapping from layer1 > layer2 > layer3 > target to Description for Field Mappings Demo Event Source 2, as in step 9.

    fm186-k-add-mapping-2.png
  4. Send the POST request.

    image-20250718-234448.png
  5. Verify that an event was created: go to Events and select All Events. Its Event Source is Field Mappings Demo Event Source 2.

    fm186-k-events-list-2.png
  6. Note the EventID of the new event.

  7. If the list shows more than one site, select the site from step 3 in the site selector.

  8. Select the EventID from step 22 to open the event’s details.

  9. Verify that the Unique Event Key is the id from the payload in step 17 (2).

    fm186-k-event2-key.png
  10. Verify that the value from the "target" payload field appears in Description, the destination field configured in step 19.

    fm186-k-event2-desc.png
Testing IFM & EFM – Search Strings, Source Field, Data Ingestion (Microsoft Sentinel)

part 1

  1. Navigate to the Fetch Incident command of the Microsoft Sentinel integration.

    Microsoft Sentinel commands list with Fetch Incident
  2. Under External Access, turn on Accept events from outside D3, then select Webhook keys.

    fm186-sent-external-access.png
  3. Go to Data Ingestion > Incident Intake, select New > Webhook, then choose the Microsoft Sentinel integration and a site. Under Webhook Keys, select Generate a Key, enter a key name, and select Generate. Note the selected site.

  4. Copy the Request URL, the Request Header Key (d3key) and the Request Header Value to Postman. Optionally, select a preprocessing playbook under Additional Settings.

    fm186-k-incident-key-request.png

    In Postman:

    • Prepare the POST request URL.

      Frame 114-20250723-190237.png
    • Prepare the request header key and value.

      Frame 115-20250723-190338.png
  5. Prepare the following payload for ingestion:

    JSON
    {
        "value": [
            {
                "discriminator": "lorem_ipsum",
                "properties": {
                    "title": "demo_title",
                    "description": "demo_description",
                    "incidentNumber": 99999
                },
                "alertDetails": [
                    {
                        "Tactics": "CredentialAccess",
                        "Techniques": "",
                        "Type": "demo_type",
                        "SystemAlertId": "3f8a1b94-b79e-4b65-beb7-2a14f0a6e5a9",
                        "Placeholder": "demo_value"
                    }
                ]
            }
        ]
    }
    

    Frame 125-20250723-220214.png
  6. Under Incident Field Mapping, select Setup Incident Field Mapping.

    The Setup Incident Field Mapping button
  7. Select the site from step 3, then select Add Source. Enter an Incident Source Name (for example, Field Mappings Demo Incident Source) and the following Search String, then select Save:

    {.discriminator}={{ipsum$}}
    

    fm186-k-incident-source-add.png
  8. With the new source selected, select each of the following system fields, enter its Source Field, and select Save.

    fm186-k-incident-edit-fields.png
  9. Verify that the IFM reflects the updated source fields.

    Incident Field Mapping for the demo incident source
  10. Under Event Field Mapping, select Setup Event Field Mapping.

    The Setup Event Field Mapping button on Fetch Incident
  11. Set the Event Root Path to $.alertDetails.

    fm186-sent-root-path.png
  12. Add a mapping set called Field Mappings Demo Event Source as in the Zendesk test: add the alertDetails part of the payload as custom sample data in the Source Field JSON Picker, pick Type, and enter demo_type as the value. The set’s condition reads:

    {$.Type}=demo_type
    

    fm186-k-sent-picker.png
    fm186-k-sent-new-set.png

    This instructs the system to evaluate the $.value.alertDetails.Type key in the payload, and apply this event source when its value equals the string "demo_type".

  13. Select Add Mapping four times to map Tactics to Tactics, Type to Event Type, SystemAlertId to Unique Event Key, and Placeholder to Description.

    Field Mappings Demo Event Source with four mappings

PROGRESS SUMMARY

  • The value array serves as the entry point for the IFM.

    • Each unique object within it (based on incidentNumber) contains data used to generate a D3 incident.

    • Three IFMs are configured.

  • The alertDetails array serves as the entry point for the EFM.

    • Each unique object within it (based on SystemAlertId) contains data used to generate a D3 event.

    • Four EFMs are configured.

      • The Description of the D3 event is extracted from the path $.value.alertDetails.Placeholder.

  1. Send the POST request prepared in step 5.

    Frame 127-20250723-221831.png
  2. Verify that a D3 incident was created: go to Incidents, select All, and note the Incident No. of the new incident from Microsoft Sentinel.

    fm186-k-incident-list.png
  3. If the list shows more than one site, select the site from step 3 in the site selector.

  4. Select the incident’s title, then select View Full Details.

  5. Verify that the incident title is the title from the payload, as configured in step 9.

    fm186-k-incident-title.png
  6. Verify that the incident’s Description also reflects the value configured in step 9.

    fm186-k-incident-desc.png
  7. In Events Summary, expand the event.

  8. Verify that the four values configured in step 13 are displayed.

    fm186-k-incident-events.png

Order Priority

When search strings from different sources simultaneously match different key-value pairs in the ingested JSON payload, only one non-default source is applied. For event mapping sets, the set higher in the Field Mapping Sets list is applied; to change the order, open the Field Mapping Sets menu (…), select Reorder, drag the sets, and select Save. For incident sources, the source with the lower Order Priority value is applied.

The Field Mapping Sets menu with Add Mapping Set and Reorder
Field Mapping Sets in reorder mode, with drag handles, Cancel and Save
fm186-k-incident-priority-edit.png
  • The (Default Event Source) mappings apply regardless of Order Priority.

  • Only one non-default event source applies per post-ingestion pipeline execution.

  • Search strings from multiple event sources may match fields within the same payload.

    • For event mapping sets, the set higher in the Field Mapping Sets list takes precedence.

    • For incident sources, the source with the lower Order Priority value takes precedence, even when it was created later.

In the example below, two incident sources match the same payload and set the title from different fields. With Order Priority 1 and 2, the title came from the first source; after the first source was changed to 3, the title came from the second.

fm186-k-incident-priority-list.png
Testing Order Priority (Zendesk)

objective – Understand the effect of order priority when search strings from different sources simultaneously match different key-value pairs in the ingested JSON payload.


  1. Add the following two mapping sets, in this order, as in the Zendesk test above: Field Mappings Demo Priority 1 with condition discriminator1 = value1, then Field Mappings Demo Priority 2 with condition discriminator2 = value2.

    fm186-k-op-set-1.png
    fm186-k-op-set-2.png

    search string 1

    {$.discriminator1}=value1
    

    search string 2

    {$.discriminator2}=value2
    
  2. Prepare the following payload for ingestion:

    Frame 145-20250724-214641.png
    JSON
    {
        "results": [
            {
                "id": "902",
                "discriminator1": "value1",
                "discriminator2": "value2",
                "target0": "(Default Event Source) matched",
                "target1": "Demo Event Source 1 matched",
                "target2": "Demo Event Source 2 matched"
            }
        ]
    }
    
  3. Add the following field mappings. The (Default Event Source) maps the id to Unique Event Key by default.

    1. Field Mappings Demo Priority 1: target1 to Description.

      fm186-k-op-mapping-1.png
    2. Field Mappings Demo Priority 2: target2 to Description.

      fm186-k-op-mapping-2.png
    3. The two sets in the Field Mapping Sets list, Field Mappings Demo Priority 1 above Field Mappings Demo Priority 2:

      fm186-k-op-sets.png
  4. Send the POST request.

    Frame 146-20250724-214836.png
  5. Verify that an event was created: go to Events and select All Events.

  6. Note the EventID of the new event.

  7. Check its Event Source: Field Mappings Demo Priority 1.

  8. If the list shows more than one site, select the site in which the key was generated.

  9. Select the EventID observed in step 6 to open the event’s details.

  10. Verify that the Unique Event Key matches the id specified in step 2, and that Description shows Demo Event Source 1 matched: both the (Default Event Source) and Field Mappings Demo Priority 1 mappings were applied.

    fm186-k-op-event.png

    Field Mappings Demo Priority 1 was applied because it is higher in the Field Mapping Sets list than Field Mappings Demo Priority 2.