Configuring Data Ingestion

LAST UPDATED: JAN 13, 2025

Configuring data ingestion is a prerequisite for automating workflows that use external data. This guide outlines the steps for configuring data ingestion using the Schedule method (pulling data into D3) and the Webhook method (pushing data into D3).

Accessing Data Ingestion Settings

1. Click on the Configuration navigational link.

2. Click on the Data Ingestion module.

3. Click on the + button.

After clicking the + button, you will see two options:

• Schedule: Data will be ingested from third-party integrations automatically.

• Webhook: Enables users to transmit data into D3 vSOC through API testing tools (e.g. Postman).

Configuring Data Ingestion

Scheduled Ingestion

1. Select an Integration (e.g. Microsoft Sentinel).

2. Select the integration Connection configured beforehand.

READER NOTE

This connection facilitates data retrieval from the selected integration. For Microsoft Sentinel configuration instructions, refer to Configuring Microsoft Sentinel to Work with D3 SOAR and Configuring D3 SOAR to Work with Microsoft Sentinel.

3. Select the Event Playbook to run after the data is ingested, within the Additional Settings accordion.

4. Set the Start Time (in UTC) for initiating data ingestion.

5. Configure integration-specific input parameters. For the case of Microsoft Sentinel, this includes entering a search query to retrieve specific data. Let us use the following Search Condition (Kusto Query Language):

   CODE

   SecurityIncident | where Status == "New"

6. Click on the Save & Run button.

Reviewing Ingestions

1. Navigate to the Data Ingestion module.

2. Select the data ingestion schedule task created. A list of ingestion actions will appear on the right.

3. Observe the ingestions.

   • (with no number beside it) indicates the absence of available data at that time.

   • (with a number to its left) signifies successful data ingestion with the count of D3 events created.

4. Click on the category tabs to access the desired data ingestion details.

READER NOTE

The D3 events created and their corresponding IDs are available in the Event List tab.

5. View specific ingestion details in the code preview section.

Webhook Ingestion

If Webhook was selected instead of Schedule, perform the following steps:

1. Select the Integration name.

2. Select the Site to share or make data available.

3. Enable the API key Authentication Method.

4. Use the API Request URL, Request Header Key, and Request Header Value to configure API clients for pushing data into D3.

5. Select the Event Playbook, within the Additional Settings accordion, to process the ingested data.