Skip to main content
Skip table of contents

Google Cloud PubSub

LAST UPDATED: 06/12/2024

Overview

Google Cloud PubSub (Publisher/Subscriber) is a fully-managed real-time messaging service that allows clients to send and receive messages between independent applications. Google Cloud PubSub integration allows clients to manage subscriptions and publish/pull topic messages.

D3 vSOC is providing REST operations for working with Google Cloud PubSub.

Google Cloud PubSub is applicative in:

D3 SOAR

V14.5.53.0+

Category

Cloud Services

Deployment Options

Option II, Option IV

Known Limitations

The table below provides the information about resource limits for Pub/Sub.

Resource

Limits

Project

10,000 topics

10,000 attached or detached subscriptions

5,000 snapshots

10,000 schemas

Topic

10,000 attached subscriptions

5,000 attached snapshots

If topic message retention is configured, messages published to a topic can be retained in persistent storage for up to 31 days from the time of publication.

Subscription

By default, retains unacknowledged messages in persistent storage for 7 days from the time of publication. There is no limit on the number of retained messages.

If subscribers don't use a subscription, the subscription expires. The default expiration period is 31 days.

Schema

Schema size (the definition field): 50KB

Publish request

10MB (total size)

1,000 messages

Message

Message size (the data field): 10MB

Attributes per message: 100

Attribute key size: 256 bytes

Attribute value size: 1024 bytes

Push outstanding messages

3,000 * N by default.

30,000 * N for subscriptions that acknowledge >99% of messages and average <1s of push request latency.

N is the number of publish regions. For more information, see Using push subscriptions.

StreamingPull streams

10 MB/s per open stream

Pull/StreamingPull messages

The service might impose limits on the total number of outstanding StreamingPull messages per connection. If you run into such limits, increase the rate at which you acknowledge messages and the number of connections you use.

Ordering keys

If messages have ordering keys, the maximum publisher throughput is 1 MBps for each ordering key.

Please refer to Pub/Sub quotas and limits for detailed information.

Connection

To connect Google Cloud PubSub from D3 SOAR, please follow this part to collect the required information below:

Parameter

Description

Example

Service Account JSON

The content of the service account JSON file. Regarding how to get the service account JSON file, please refer to https://developers.google.com/identity/protocols/oauth2/service-account. Note: If the service account uses the built-in role, then the service account must have one of the following roles: Project-Owner, Project-Editor, Pub/Sub Admin, Pub/Sub Editor (Recommended). For best security practice, you may also create a custom role that includes the required permissions for the commands as below: Test Connection (pubsub.topics.list), Acknowledge Messages (pubsub.subscriptions.consume), Create Subscription (pubsub.subscriptions.create, pubsub.topics.attachSubscription), Fetch Event (pubsub.subscriptions.consume), List Subscriptions (pubsub.subscriptions.list), List Topics (pubsub.topics.list), Publish Message (pubsub.topics.publish)

{

"type": "srvice_account",

"project_id": "nimble-cortex-*****",

"private_key_id": "*****",

"private_key": "*****",

"client_email": "******@*****.***",

"client_id": "*****",

"auth_uri": "https://accounts.google.com/*****",

"token_uri": "https://oauth2.googleapis.com/*****",

"auth_provider_x509_cert_url": "https://www.googleapis.com/*****",

"client_x509_cert_url": "https://www.googleapis.com/*****"

}

Permission Requirements

Each endpoint in Google Cloud PubSub API requires a certain permission scope. The following scopes are required scopes for each command in this integration:

Command

Google Cloud prefixed Pub/Sub IAM roles

Custom Role Required Permissions

Pub/Sub Publisher

Pub/Sub Subscriber

Pub/Sub Viewer

Pub/Sub Editor

Pub/Sub

Admin

Acknowledge Messages

🗙

✔

🗙

✔

✔

pubsub.subscriptions.consume

Create Subscription

🗙

🗙

🗙

✔

✔

pubsub.subscriptions.create

pubsub.topics.attachSubscription

Fetch Event

🗙

✔

🗙

✔

✔

pubsub.subscriptions.consume

List Subscriptions

🗙

🗙

✔

✔

✔

pubsub.subscriptions.list

List Topics

🗙

🗙

✔

✔

✔

pubsub.topics.list

Publish Message

✔

🗙

🗙

✔

✔

pubsub.topics.publish

Test Connection

🗙

🗙

✔

✔

✔

pubsub.topics.list

READER NOTE

The five pre-configured roles in Google Pub/Sub IAM address many typical use cases. You may also create a custom role that includes the listed permissions above for D3 SOAR commands execution. To get more information about Required Permissions, please refer to Cloud Pub/Sub Access control with IAM.

Configuring Google Cloud PubSub to Work with D3 SOAR

Obtaining the Service Account JSON

  1. To connect Google Kubernetes Engine with D3 SOAR, we need to configure the Service Account JSON file. First, log in to the Google Cloud Platform (GCP) console with admin credentials.

  2. Click the Hamburger Menu in the top left corner to reveal the sidebar menu. Navigate to APIs and services. In its submenu, select Credentials.

  3. Click + CREATE CREDENTIALS, and select Service account.

  4. Under Service account details, type a name, ID, and description for the service account, then click CREATE AND CONTINUE.

  5. Assign the desired role to the service account (e.g. Owner), then click CONTINUE.

  6. (Optional) Grant users access to this service account if desired, then click DONE.

  7. You will find the service account you have just created on the Credentials page. Click and open the service account.

  8. In the KEYS tab, click ADD KEY, then select Create new key.

  9. Choose JSON as the key type, then click CREATE.

  10. The Service Account JSON file (Private key) will automatically be downloaded to your computer.

  11. If it is your first time using the Google PubSub API, you will need to enable it. First, click the Google Cloud Platform navigation menu, in the slider sidebar menu, find API and Services, hover over it, and then in its submenu, select Library.

  12. Search and select Cloud Pub/Sub API, then click ENABLE.

Configuring the Service Account Domain-wide Delegation

You will also need to enable the Google Workspace domain-wide delegation for your created service account. Please follow the steps below.

  1. Log in to the Google Workspace Admin Console with admin credentials. Click Security > Access and data control > API controls. Scroll down and click MANAGE DOMAIN-WIDE DELEGATION.

  2. Click Add new to add a new API client.

  3. Find your Client ID in the service account you created and paste it into the Client ID field. Input https://www.googleapis.com/auth/cloud-platform into the OAuth scopes field, then click AUTHORISE.

  4. The service account domain-wide delegation can now be found on the API controls page. Your created service account is now ready to use.

Creating custom roles (Optional)

READER NOTE

It is best practice to grant only the necessary permissions when creating roles. You can do so by configuring a custom role with essential permissions.

A custom role, with essential permissions, is suggested if you are planning to use the google cloud default kubernetes roles. Please skip this section and start from step 7 of Obtaining the Service Account JSON.

  1. Log in to the Google Cloud Platform (GCP) console with admin credentials.

  2. Click on the hamburger menu found on the top left corner to reveal the sidebar menu. Click IAM and admin, then Roles.

  3. Click + CREATE ROLE on the top bar menu.

  4. Input Title, and click the button ADD PERMISSIONS. Search and select the desired permissions, then click ADD. Click CREATE.

READER NOTE

The following six permissions must be included to execute all Google Pub/Sub integration commands in D3 SOAR.

Please refer to the Permission Requirements section for the required permissions associated with each Google Cloud PubSub integration command in D3 SOAR.

Assigning Permissions to a Service Account

  1. Log in to your GCP portal. From the left sidebar menu, navigate to IAM & Admin > IAM. Under the PERMISSION tab, choose a service account, then click the pencil icon.

  2. Make sure to add one of the following roles, and click SAVE.

    • Pub/Sub Editor (Recommended)

    • Pub/Sub Viewer

    • Pub/Sub Subscriber

    • Pub/Sub Publisher

    • Pub/Sub Admin

    • Custom role for Google Cloud PubSub access (Refer to Creating custom roles (Optional) for the role configuration; check Permission Requirements for the minimum permission for each command).

Configuring D3 SOAR to Work with Google Cloud PubSub

  1. Log in to D3 SOAR.

  2. Find the Google Cloud PubSub integration.

    1. Navigate to Configuration on the top header menu.

    2. Click on the Integration icon on the left sidebar.

    3. Type Google Cloud PubSub in the search box to find the integration, then click it to select it.

    4. Click + Connection, on the right side of the Connections section. A new connection window will appear.

  3. Configure the following fields to create a connection to Google Cloud PubSub.

    1. Connection Name: The desired name for the connection.

    2. Site: Specifies the site to use the integration connection. Use the drop-down menu to select the site. The Share to Internal Sites option enables all sites defined as internal sites to use the connection. Selecting a specific site will only enable that site to use the connection.

    3. Recipient site for events from connections Shared to Internal Sites: This field appears if you selected Share to Internal Sites for Site to let you select the internal site to deploy the integration connection.

    4. Agent Name (Optional): Specifies the proxy agent required to build the connection. Use the dropdown menu to select the proxy agent from a list of previously configured proxy agents.

    5. Description (Optional): Add your desired description for the connection.

    6. Tenant (Optional): When configuring the connection from a master tenant site, you have the option to choose the specific tenant sites you want to share the connection with. Once you enable this setting, you can filter and select the desired tenant sites from the dropdowns to share the connection.

    7. Configure User Permissions: Defines which users have access to the connection.

    8. Active: Check the tick box to ensure the connection is available for use.

    9. System: This section contains the parameters defined specifically for the integration. These parameters must be configured to create the integration connection.
      1. Paste your Server Account JSON. Please refer to step 10 of Obtaining the Service Account JSON to get the value.

    10. Connection Health Check: Updates the connection status you have created. A connection health check is done by scheduling the Test Connection command of this integration. This can only be done when the connection is active.
      To set up a connection health check, check the Connection Health Check tickbox. You can customize the interval (minutes) for scheduling the health check. An email notification can be set up after a specified number of failed connection attempts.

    11. Enable Password Vault: An optional feature that allows users to take the stored credentials from their own password vault. Please refer to the password vault connection guide if needed.

  4. Test the connection.

    1. Click Test Connection to verify the account credentials and network connection. If the Test Connection Passed alert window appears, the test connection is successful. You will see Passed with a green checkmark appear beside the Test Connection button. If the test connection fails, please check your connection parameters and try again.

    2. Click OK to close the alert window.

    3. Click + Add to create and add the configured connection.

Commands

Google Cloud PubSub includes the following executable commands for users to set up schedules or create playbook workflows. With the Test Command, you can execute these commands independently for playbook troubleshooting.

Integration API Note

For more information about the Google Cloud PubSub API, please refer to the Google Cloud PubSub API reference.

READER NOTE

Certain permissions are required for each command. Please refer to the Permission Requirement and Configuring Google Cloud PubSub to Work with D3 SOAR for details.

Note for Time-related parameters

The input format of time-related parameters may vary based on your account settings. As a result, the sample data provided in our commands is different from what you see. To set your preferred time format, follow these steps:

  1. Navigate to Configuration > Application Settings. Select Date/Time Format.

  2. Choose your desired date and time format.

After that, you will be able to view your preferred time format when configuring the DateTime input parameters for commands.

Acknowledge Messages

Acknowledges the messages associated with the ACK IDs. The Pub/Sub system can remove the relevant messages from the subscription.

READER NOTE

Subscription ID and ACK IDs are required parameters to run this command.

  • Run the List Subscriptions command to obtain the Subscription ID. The Subscription ID is the last part of the subscription’s name. Subscription names can be found in the raw data at the path $.subscriptions[*].name. For example, if the subscription’s name is projects/nimble-cortex-285618/subscriptions/subAPI_1109a, the Subscription ID is subAPI_1109a.

  • Run the Fetch Event command to obtain the ACK IDs. ACK IDs can be found in the raw data at the path $.received_messages[*].ackId.

Input

Input Parameter

Required/Optional

Description

Example

Subscription ID

Required

The ID of the subscription associated with the message being acknowledged. Subscription ID can be obtained using the List Subscriptions command.

sub*****08b

ACK IDs

Required

The acknowledgment IDs of the messages being acknowledged that were returned by the Pub/Sub system in the subscriptions. ACK ID can be obtained using the Fetch Event command.

[ "RFA*****FUw" ]

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful
Raw Data

The primary response data from the API request.

SAMPLE DATA

CODE 
{
    "actionResult": "Acknowledged the messages successfully"
}
Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

actionResult

Acknowledged the messages successfully

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

Acknowledge Messages failed. An error occurred when calling the Acknowledge Messages operation.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 400.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: You have passed a subscription that does not belong to the given ack ID.

Error Sample Data

Acknowledge Messages failed. An error occurred when calling the Acknowledge Messages operation.

Status Code: 400.

Message: You have passed a subscription that does not belong to the given ack ID.

Create Subscription

Creates a subscription to a given Topic.

READER NOTE

Topic ID is a required parameter to run this command.

  • Run the List Topics command to obtain Topic ID. The Topic ID is the last part of the topic name. Topic names can be found in the raw data at the path $.topics[*].name. For example, if the topic name is projects/nimble-cortex-285618/subscriptions/TestTopicAPI1101, the Topic ID is TestTopicAPI1101.

  • Cloud Pub/Sub can automatically delete a subscription after a specified period if there is no subscriber activity such as open connections, active pulls or successful pushes. This is useful for creating temporary subscriptions. This period must be longer than the message retention duration. Users can set Expiration TTL on D3 SOAR to set subscription duration.

Input

Input Parameter

Required/Optional

Description

Example

Subscription Name

Required

The name of the subscription to be created. Note: The subscription name must be between 3-255 characters, starts with a letter, and contains only the following characters: letters, numbers, dashes (-), periods (.), underscores (_), tildes (~), percents (%) or plus signs (+). Additionally, a subscription name cannot start with "goog".

subAPI_*****

Topic

Required

The Topic ID to which the subscription is created.

TestTopic*******

Push Endpoint

Optional

If push delivery is using this subscription, pushEndpoint should be specified. pushEndpoint is the URL locating the endpoint where messages should be pushed. If not specified, Pull is the default delivery type for the subscription.

https://****.testsoar.com/********

ACk Deadline Seconds

Optional

The approximate amount of time (on a best effort basis) the Pub/Sub waits for the subscriber to acknowledge receipt before resending the message. The valid value range is between 10 to 600 seconds. The default is set to 10 seconds if the parameter is not specified.

30

Retain Acked Messages

Optional

Indicates whether to retain acknowledged messages. The default value is false. Enabling this option may increase message storage fees.

True

Message Retention Duration (seconds)

Optional

The duration to retain unacknowledged messages in the subscription's backlog, from the moment a message is published. Defaults to 7 days(604800 seconds). The value of this field needs to be greater than 10 minutes and less than 7 days.If not specified, the default value 604800 will be used.

604800

Labels

Optional

The labels to be added to the created subscription. The format is KeyName=KeyValue pairs separated by the space character. For example, name=test_subscription id=*****. If not specified, no label will be used.

name=test_subscription id=*****

Expiration TTL (seconds)

Optional

The "time-to-live" duration in seconds for the subscription. If it is NOT specified, the subscription will never expire. If set an expiration, the minimum value is 86400(1 day), and it must be longer than the message retention duration.

86400

Output

Return Data

Indicates one of the possible command execution states: Successful, Partially Successful, or Failed.

The Partially Successful state only occurs when a command’s input accepts an array of items (e.g. an array of IP addresses) and one or more items within the array return an error from the API request.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful
Raw Data

The primary response data from the API request.

SAMPLE DATA

CODE 
{
    "name": "projects/nimble-cortex-285***/subscriptions/subAPI_*********",
    "topic": "projects/nimble-cortex-285***/topics/TestTopic*******",
    "push_config": {
        "push_endpoint": "https://demo.d3soar.com/*****",
        "attributes": {
            "x-goog-version": "v1"
        }
    },
    "ack_deadline_seconds": 300,
    "retain_acked_messages": true,
    "message_retention_duration": {
        "seconds": 604800
    },
    "labels": {
        "name": "test_subscription",
        "id": "*****"
    },
    "expiration_policy": {
        "ttl": {
            "seconds": "2678400s"
        }
    }
}
Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.

The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE 
{
    "SubscriptionName": [
        "projects/nimble-cortex-285***/subscriptions/subAPI_*********"
    ]
}
Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

name

projects/nimble-cortex-285618/subscriptions/subAPI_1109aPush

topic

projects/nimble-cortex-285618/topics/TestTopicAPI1101

push_config

{'push_endpoint': 'https://demo.d3soar.com/*****', 'attributes': {'x-goog-version': 'v1'}}

ack_deadline_seconds

300

retain_acked_messages

True

message_retention_duration

{'seconds': 604800}

labels

{'name': 'test_subscription', 'id': '*****'}

expiration_policy

{'ttl': {'seconds': '2678400s'}}

Error Handling

If the Return Data is Partially Successful or Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

Create Subscription failed. An error occurred (TopicIDNotFound) when calling the Create Subscription operation.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 404.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: Topic ID Not Found.

Error Sample Data

Create Subscription failed. An error occurred (TopicIDNotFound) when calling the Create Subscription operation.

Status Code: 404.

Message: Topic ID Not Found

Fetch Event

Pulls message(s) from the specified PubSub subscription which is based on the message(s) publish time.

READER NOTE

Subscription ID is a required parameter to run this command.

  • Run the List Subscriptions command to obtain the Subscription ID. The Subscription ID is the last part of the subscription’s name. Subscription Names can be found in the raw data at the path $.subscriptions[*].name. For example, if the subscription’s name is projects/nimble-cortex-285***/subscriptions/subAPI_1109a, the Subscription ID is subAPI_1109a.

Input

Input Parameter

Required/Optional

Description

Example

Start Time

Optional

The Start Time of the publish time range for fetching event(s) which are published after the specified start time in UTC time.

2021-11-01 00:00

End Time

Optional

The End Time of the publish time range for fetching event(s) which are published before the specified end time in UTC time.

2021-11-08 00:00

Top Recent Event Number

Optional

The maximum number of messages to return.

100

Subscription ID

Required

The ID of the subscription where messages are pulled. Subscription ID can be obtained using the List Subscriptions command.

TestSubscription

Acknowledge Message

Optional

Whether to acknowledge the message pulled. The default value of this field is True.

True

Ignore Time Arrange

Optional

Whether to ignore the Start Time and End Time parameters. The default value of this field is True.

True

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful
Raw Data

The primary response data from the API request.

SAMPLE DATA

CODE 
{
    "received_messages": [
        {
            "ack_id": "*****",
            "message": {
                "data": "*****",
                "attributes": {
                    "email": "*****@d3cyberlab.com",
                    "id": "*****",
                    "name": "d3 *****"
                },
                "message_id": "*****",
                "publish_time ": {
                    "seconds": 1637975668
                },
                "publishTime": "2021-11-27T01:14:28Z"
            }
        }
    ]
}
Context Data

The data extracted from Raw Data converted into JSON format. Context Data may be identical to Raw Data in some cases.

D3 customizes the Context Data by extracting the data from path $.received_messages in API returned JSON.

It is recommended to refer to the Raw Data instead of Context Data, since it contains the complete API response data. D3 will deprecate Context Data in the future, and playbook tasks using Context Data will be replaced with Raw Data.

SAMPLE DATA

CODE 
No Sample Data
Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.

The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE 
{
    "MessageIDs": [
        "*****"
    ],
    "AckIDs": [
        "*****"
    ]
}
Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

Start Time (UTC)

2023-02-07 12:32:41

End Time (UTC)

2023-02-07 12:37:41

EventsCount

2

Fetch Event Field Mapping

Please note that Fetch Event commands require event field mapping. Field mapping plays a key role in the data normalization process part of the event pipeline. Field mapping converts the original data fields from the different providers to the D3 fields which are standardized by the D3 Model. Please refer to Event and Incident Intake Field Mapping for details.

To customize field mapping, click + Add Field and add the custom field of your choice. You can also remove built-in field mappings by clicking x. Please note that two underscore characters will automatically prefix the defined Field Name as the System Name for a custom field mapping. Additionally, if an input Field Name contains any spaces, they will automatically be replaced with underscores for the corresponding System Name.

As a system integration, the Google Cloud PubSub integration has some pre-configured field mappings for default field mapping.

  • Default Event Source
    The Default Event Source is the default set of field mappings that are applied when this fetch event command is executed. For out-of-the-box integrations, you will find a set of field mapping provided by the system. Default event source provides field mappings for common fields from fetched events. The default event source has a “Main Event JSON Path” (i.e., $.received_messages) that is used to extract a batch of events from the response raw data. Click Edit Event Source to view the “Main Event JSON Path”.

    • Main Event JSON Path: $.data
      The Main Event JSON Path determines the root path where the system starts parsing raw response data into D3 event data. The JSON path begins with $, representing the root element. The path is formed by appending a sequence of child elements to $, each separated by a dot (.). Square brackets with nested quotation marks ([‘...’]) should be used to separate child elements in JSON arrays.
      For example, the root node of a JSON Path is received_messages. The child node denoting the Unique Event Key field would be message.message_id. Putting it together, the JSON Path expression to extract the Unique Event Key is $.received_messages.message.message_id.

The pre-configured field mappings are detailed below:

Field Name

Source Field

Unique Event Key

.message.message_id

Start Time

.message.publishTime

Event Type

*Google Cloud PubSub Message

Acknowledgment ID

.ack_id

*Google Cloud PubSub Message

In D3 SOAR, the events from Google Cloud PubSub will be prefilled with Google Cloud PubSub Message as the Event Type.

Please note that the source type for Event Type is defined as Placeholder, Google Cloud PubSub Message is a default mapping value provided by D3.

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

Fetch Event failed. An error occurred when calling the Fetch Event operation.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 404.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: Subscription ID Not Found

Error Sample Data

Fetch Event failed. An error occurred when calling the Fetch Event operation.

Status Code: 404.

Message: Subscription ID Not Found.

List Subscriptions

Lists all of the Cloud Pub/Sub subscriptions.

Input

N/A

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful
Raw Data

The primary response data from the API request.

SAMPLE DATA

CODE 
{
    "subscriptions": [
        {
            "name": "projects/nimble-cortex-285***/subscriptions/D3*********",
            "topic": "projects/nimble-cortex-285***/topics/TestTopic",
            "push_config": {
                "push_endpoint": "https://demo.d3soar.com/*****",
                "attributes": {
                    "x-goog-version": "v1"
                }
            },
            "ack_deadline_seconds": 10,
            "message_retention_duration": {
                "seconds": 604800
            },
            "expiration_policy": {
                "ttl": {
                    "seconds": 604800
                }
            }
        }
    ]
}
Context Data

The data extracted from Raw Data converted into JSON format. Context Data may be identical to Raw Data in some cases.

D3 customizes the Context Data by extracting the data from path $.subscriptions in API returned JSON.

It is recommended to refer to the Raw Data instead of Context Data, since it contains the complete API response data. D3 will deprecate Context Data in the future, and playbook tasks using Context Data will be replaced with Raw Data.

SAMPLE DATA

CODE 
[
    {
        "name": "projects/nimble-cortex-285***/subscriptions/D3Webhook",
        "topic": "projects/nimble-cortex-285***/topics/TestTopic",
        "push_config": {
            "push_endpoint": "https://demo.d3soar.com/*****",
            "attributes": {
                "x-goog-version": "v1"
            }
        },
        "ack_deadline_seconds": 10,
        "message_retention_duration": {
            "seconds": 604800
        },
        "expiration_policy": {
            "ttl": {
                "seconds": 604800
            }
        }
    }
]
Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.

The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE 
{
    "SubscriptionNames": [
        "projects/nimble-cortex-285***/subscriptions/subAPI_1101b"
    ],
    "SubscriptionTopics": [
        "projects/nimble-cortex-285***/topics/TestTopicAPI1101"
    ],
    "AckDeadlineSeconds": [
        "10"
    ],
    "MessageRetentionDurationSeconds": [
        "604800s"
    ]
}
Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

name

projects/nimble-cortex-285618/subscriptions/D3Webhook

topic

projects/nimble-cortex-285618/topics/TestTopic

push_config

{'push_endpoint': 'https://demo.d3soar.com/*****', 'attributes': {'x-goog-version': 'v1'}}

ack_deadline_seconds

10

message_retention_duration

{'seconds': 604800}

expiration_policy

{'ttl': {'seconds': 604800}}

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

List Subscriptions Failed. An error occurred when calling the List Subscriptions operation.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 404.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: Requested project not found or user does not have access to it.

Error Sample Data

List Subscriptions Failed. An error occurred when calling the List Subscriptions operation.

Status Code: 404.

Message: Requested project not found or user does not have access to it.

List Topics

Lists all Cloud Pub/Sub topics.

READER NOTE

This command only shows the name of the related Topics. Some other commands may require Topics ID from the result by running this command. Actually, the last part of the Topics name is the Topic ID. For example, if the topic name is projects/nimble-cortex-285618/topics/TestTopic, the Topic ID is TestTopic.

Input

N/A

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful
Raw Data

The primary response data from the API request.

SAMPLE DATA

CODE 
{
    "topics": [
        {
            "name": "projects/nimble-cortex-285618/topics/TestTopic"
        },
        {
            "name": "projects/nimble-cortex-285618/topics/TestTopicAPI1101"
        },
        {
            "name": "projects/nimble-cortex-285618/topics/TestTopicAPI1108a"
        }
    ]
}
Context Data

The data extracted from Raw Data converted into JSON format. Context Data may be identical to Raw Data in some cases.

D3 customizes the Context Data by extracting the data from path $.topics in API returned JSON.

It is recommended to refer to the Raw Data instead of Context Data, since it contains the complete API response data. D3 will deprecate Context Data in the future, and playbook tasks using Context Data will be replaced with Raw Data.

SAMPLE DATA

CODE 
[
    {
        "name": "projects/nimble-cortex-285618/topics/TestTopic"
    },
    {
        "name": "projects/nimble-cortex-285618/topics/TestTopicAPI1101"
    },
    {
        "name": "projects/nimble-cortex-285618/topics/TestTopicAPI1108a"
    }
]
Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.

The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE 
{
    "TopicNames": [
        "projects/nimble-cortex-285618/topics/TestTopic",
        "projects/nimble-cortex-285618/topics/TestTopicAPI1101"
        "projects/nimble-cortex-285618/topics/TestTopicAPI1108a"
    ]
}
Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

name

projects/nimble-cortex-285618/topics/TestTopic

projects/nimble-cortex-285618/topics/TestTopicAPI1101

projects/nimble-cortex-285618/topics/TestTopicAPI1108a

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

List Topics Failed. An error occurred when calling the List Topics operation.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 404.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: Requested project not found or user does not have access to it.

Error Sample Data

List Topics Failed. An error occurred when calling the List Topics operation.

Status Code: 404.

Message: Requested project not found or user does not have access to it.

Publish Message

Publishes one or more messages to the specified topic.

READER NOTE

Topic ID is a required parameter to run this command.

  • Run the List Topics command to obtain the Topic ID. The Topic ID is the last part of the topic name. Topic names can be found in the returned raw data at the path $.topics[*].name. For example, if the topic name is projects/nimble-cortex-285618/subscriptions/TestTopicAPI1101, the Topic ID is TestTopicAPI1101.

Messages published on the topic which has no subscription will be lost. Add a subscription or set retention to the topic to retain messages.

Input

Input Parameter

Required/Optional

Description

Example

Topic ID

Required

The ID of the topic to which the message is published. Topic ID can be obtained using the List Topics command.

TestTopic*******

Message Data

Optional

The message data field. If this field is empty, the message attributes field must contain at least one attribute.

[ "*****" ]

Message Attributes

Optional

The message attributes field. If this field is empty, the message data must not be empty. The format is KeyName=KeyValue pairs, with each attribute separated by the space character. For example, name="*****" email=*****@demo.com id=*****.

Each message can have a maximum of 100 attributes. Attributes key should not start with “goog” and must not exceed 256 bytes in size. Attributes value must not exceed 1024 bytes.

name="*****" email=*****@demo.com id=*****

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful
Raw Data

The primary response data from the API request.

SAMPLE DATA

CODE 
{
    "messageIds": [
        ""*****""
    ]
}
Context Data

The data extracted from Raw Data converted into JSON format. Context Data may be identical to Raw Data in some cases.

D3 customizes the Context Data by extracting the data from path $.topics in API returned JSON.

It is recommended to refer to the Raw Data instead of Context Data, since it contains the complete API response data. D3 will deprecate Context Data in the future, and playbook tasks using Context Data will be replaced with Raw Data.

SAMPLE DATA

CODE 
{
    "messageIds": [
        ""*****""
    ]
}
Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.

The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE 
{
    "MessageIDs": [
        ""*****""
    ]
}
Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

messageIds

  • "*****"

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

Publish Message failed Failed. An error occurred (TopicIDNotFound) when calling the Publish Message failed operation.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 404.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: Topic ID Not Found.

Error Sample Data

Publish Message failed Failed. An error occurred (TopicIDNotFound) when calling the Publish Message failed operation.

Status Code: 404.

Message: Topic ID Not Found.

Test Connection

Allows you to perform a health check on an integration connection. You can schedule a periodic health check by selecting Connection Health Check when editing an integration connection.

Input

N/A

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

  • A connection issue with the integration

  • The API returned an error message

  • No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE 
Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error

Description

Example

Failure Indicator

Indicates the command failure that happened at a specific input and/or API call.

Test Connection failed. Failed to check the connector.

Status Code

The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Google Cloud PubSub portal. Refer to the HTTP Status Code Registry for details.

Status Code: 404.

Message

The raw data or captured key error message from the integration API server about the API request failure.

Message: Requested project not found or user does not have access to it.

Error Sample Data

Test Connection failed. Failed to check the connector.

Status Code: 404.

Message: Requested project not found or user does not have access to it.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close