Recorded Future-SecurityTrails

Overview

SecurityTrails is a total inventory that curates comprehensive domain and IP address data for users and applications that demand clarity. By combining current and historic data of all Internet assets, SecurityTrails is the proven solution for 3rd-party risk assessment, attack surface reduction and threat hunting. This integration enables organizations to query risk issues and correlate domain and IP address data for risk assessment and threat hunting.

D3 SOAR is providing REST operations to function with Recorded Future-SecurityTrails.

Recorded Future-SecurityTrails is available for use in:

D3 SOAR	V15.4.41.0+
Category	Data Enrichment
Deployment Options	Option II, Option IV

Known Limitations

It is important to monitor your account quota, as exceeding your credit limit will prevent commands from running. You can edit your subscription to purchase additional credits if you anticipate needing a higher quota.

Connection

To connect to Recorded Future-SecurityTrails from D3 SOAR, please follow this part to collect the required information below:

Parameter	Description	Example
Server URL	The server URL of the SecurityTrails API.	https://api.securitytrails.com
API Key	The API key to authenticate the connection.	ur*9Z7S*nk
API Version	The version of the API to use for the connection.	v1

Permission Requirements

No permissions are required to create API keys in SecurityTrails.

Configuring Recorded Future-SecurityTrails to Work with D3 SOAR

Log in to your SecurityTrails account using your username and password.
Locate the API section on the left navigation menu and click on API Keys.
Next, select Create new API Key and enter a name for the key. Click Create new API Key to proceed.
Your new API key will now be visible in the list of API keys. Store this key securely to configure an integration connection with D3 SOAR.

Configuring D3 SOAR to Work with Recorded Future-SecurityTrails

Log in to D3 SOAR.
Find the Recorded Future-SecurityTrails integration.
1. Navigate to Configuration on the top header menu.
2. Click on the Integration icon on the left sidebar.
3. Type Recorded Future-SecurityTrails in the search box to find the integration, then click it to select it.
4. Click + New Connection, on the right side of the Connections section. A new connection window will appear.
Configure the following fields to create a connection to Recorded Future-SecurityTrails.
1. Connection Name: The desired name for the connection.
2. Site: Specifies the site to use the integration connection. Use the drop-down menu to select the site. The Share to Internal Sites option enables all sites defined as internal sites to use the connection. Selecting a specific site will only enable that site to use the connection.
3. Recipient site for events from connections Shared to Internal Sites: This field appears if you selected Share to Internal Sites for Site to let you select the internal site to deploy the integration connection.
4. Agent Name (Optional): Specifies the proxy agent required to build the connection. Use the dropdown menu to select the proxy agent from a list of previously configured proxy agents.
5. Description (Optional): Add your desired description for the connection.
6. Configure User Permissions: Defines which users have access to the connection.
7. Active: Check the tick box to ensure the connection is available for use.
8. System: This section contains the parameters defined specifically for the integration. These parameters must be configured to create the integration connection.
  1. Input your domain level Server URL. The default value is https://api.securitytrails.com.
  2. Input your API Key. See Step 4 of Configuring Recorded Future-SecurityTrails to Work with D3 SOAR for instructions on obtaining the API Key.
  3. Input your API Version. The default value is v1.
9. Connection Health Check: Updates the connection status you have created. A connection health check is done by scheduling the Test Connection command of this integration. This can only be done when the connection is active.
  To set up a connection health check, check the Connection Health Check tickbox. You can customize the interval (minutes) for scheduling the health check. An email notification can be set up after a specified number of failed connection attempts.
10. Enable Password Vault: An optional feature that allows users to take the stored credentials from their own password vault. Please refer to the password vault connection guide if needed.
Test the connection.
1. Click Test Connection to verify the account credentials and network connection. If the Test Connection Passed alert window appears, the test connection is successful. You will see Passed with a green checkmark appear beside the Test Connection button. If the test connection fails, please check your connection parameters and try again.
2. Click OK to close the alert window.
3. Click + Add to create and add the configured connection.

Commands

Recorded Future-SecurityTrails includes the following executable commands for users to set up schedules or create playbook workflows. With the Test Command, you can execute these commands independently for playbook troubleshooting.

Integration API Note

For more information about the Recorded Future-SecurityTrails API, please refer to the following SecurityTrails API references:

DSL Query

Runs a Domain Specific Language (DSL) query on the SecurityTrails Exploration End Point platform. This command lets you query SecurityTrails Exploration End Point with flexible SQL-like queries. The syntax used for SecurityTrails' DSL closely resembles that of SQL WHERE predicates.

Input

Input Parameter	Required/Optional	Description	Example
Query	Required	The Doman Specific Language (DSL) statement to run the query. The syntax used for SecurityTrails' DSL closely resembles that of SQL WHERE predicates. For more information, such as the available input fields and sample queries, see How to use the DSL.	ns = 'ns1.yahoo.com' AND ipv4 = '1.1.1.1'

Output

Raw Data

The primary response data from the API request.

SAMPLE DATA

JSON

{
    "endpoint": "/v1/search/list",
    "meta": {
        "max_page": 1,
        "page": 1,
        "query": "ns = 'https://*****.com/***' AND ipv4 = '***.***.***.***'",
        "total_pages": 1
    },
    "record_count": 2,
    "records": [
        {
            "alexa_rank": 10,
            "computed": {
                "company_name": "Yahoo! Inc."
            },
            "host_provider": [
                "Oath Holdings Inc."
            ],
            "hostname": "https://*****.com/***",
            "ips": [
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***"
            ],
            "mail_provider": [
                "Oath Holdings Inc."
            ],
            "whois": {
                "createdDate": 790416000,
                "expiresDate": 1674104400,
                "registrar": "MarkMonitor, Inc."
            }
        },
        {
            "alexa_rank": null,
            "computed": {
                "company_name": "Domains By Proxy, LLC"
            },
            "host_provider": [
                "Oath Holdings Inc."
            ],
            "hostname": "https://*****.com/***",
            "ips": [
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***"
            ],
            "mail_provider": [
                "Oath Holdings Inc."
            ],
            "whois": {
                "createdDate": 1581610540,
                "expiresDate": 1676304940,
                "registrar": "Alibaba Cloud Computing (Beijing) Co., Ltd."
            }
        }
    ]
}

Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.
The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE

{
    "RecordCount": 2
}

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

A connection issue with the integration
The API returned an error message
No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE

Successful

Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

max_page	1
page	1
query	ns = 'ns1.yahoo.com' AND ipv4 = '1.1.1.1'
total_pages	1

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error	Description	Example
Failure Indicator	Indicates the command failure that happened at a specific input and/or API call.	DSL Query failed.
Status Code	The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Recorded Future-SecurityTrails portal. Refer to the HTTP Status Code Registry for details.	Status Code: 403.
Message	The raw data or captured key error message from the integration API server about the API request failure.	Message: Invalid Query.

Error Sample Data

DSL Query failed.

Status Code: 403.

Message: Invalid Query.

Get Current Issues

Retrieves current risk issues of the specified project.

Reader Note

Project ID is a required parameter to run this command.

In SurfaceBrowser, click on the Project tab located in the top right-hand corner of the screen. From here, you can either create a new project or select an existing one. For instance, you may choose to select the "D3 Security" project.
Click on the project to view its details. To obtain the project ID, you can look at the URL of the project page and find the alphanumeric code following the "project/" segment of the URL.

Input

Input Parameter	Required/Optional	Description	Example
Project ID	Required	The ID of the project to retrieve risk issues. Project IDs can be obtained from the SurfaceBrowser Web user interface by navigating to the project section and selecting the desired project. The Project ID can be found in the URL of the project page.	eb*****b6

Output

Raw Data

The primary response data from the API request.

SAMPLE DATA

JSON

{
  "data": [
      {
          "classification": "informational",
          "description": "2 hosts seen supporting Basic Authentication",
          "example_entities": {
              "domains": [
                  {
                      "additional": "Basic realm=\"d3secstage\"",
                      "example": "https://*****.com/***",
                      "sort_value": 60
                  },
                  {
                      "additional": "Basic realm=\"d3secdev\"",
                      "example": "https://*****.com/***",
                      "sort_value": 60
                  }
              ]
          },
          "id": "***************************************",
          "name": "Hosts with Basic Authentication",
          "rule_metadata": {
              "entity_counts": {
                  "domains": 2,
                  "ips": 0
              },
              "references": []
          }
      },
      {
          "classification": "informational",
          "description": "2 potential development servers are addressable on the public internet",
          "example_entities": {
              "domains": [
                  {
                      "additional": "Staging",
                      "example": "https://*****.com/***",
                      "sort_value": 35
                  },
                  {
                      "additional": "Development",
                      "example": "https://*****.com/***",
                      "sort_value": 35
                  }
              ]
          },
          "id": "***************************************",
          "name": "Development Infrastructure Exposure",
          "rule_metadata": {
              "entity_counts": {
                  "domains": 2,
                  "ips": 0
              },
              "references": []
          }
      },
      {
          "classification": "informational",
          "description": "1 hosts seen using a certificate where subject and issuer names are the same.",
          "example_entities": {
              "domains": [
                  {
                      "additional": "Issuer: Kubernetes Ingress Controller Fake Certificate",
                      "example": "https://*****.com/***",
                      "sort_value": 35
                  }
              ]
          },
          "id": "***************************************",
          "name": "Hosts with Self-Signed SSL/TLS Certificates",
          "rule_metadata": {
              "entity_counts": {
                  "domains": 1,
                  "ips": 0
              },
              "references": []
          }
      }
  ],
  "meta": {
      "counts": {
          "informational": 3
      },
      "project_id": "***-***-***-***-***",
      "project_title": "D3 Security",
      "snapshot": "2022-12-15 21:44:03"
  }
}

Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.
The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE

{
    "Names": [
        "Hosts with Basic Authentication"
    ],
    "IDs": [
        "crawl_basic-authentication"
    ],
    "Classifications": [
        "informational"
    ],
    "Descriptions": [
        "2 hosts seen supporting Basic Authentication"
    ],
    "DomainCounts": [
        2
    ],
    "IPCounts": [
        0
    ]
}

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

A connection issue with the integration
The API returned an error message
No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE

Successful

Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

informational

3

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error	Description	Example
Failure Indicator	Indicates the command failure that happened at a specific input and/or API call.	Get Current Issues failed.
Status Code	The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Recorded Future-SecurityTrails portal. Refer to the HTTP Status Code Registry for details.	Status Code: 404.
Message	The raw data or captured key error message from the integration API server about the API request failure.	Message: Project ID Not Found.

Error Sample Data

Get Current Issues failed.

Status Code: 404.

Message: Project ID Not Found.

Get Issue History

Retrieves a risk issue's history based on the specified criteria.

Reader Note

Project ID is a required parameter to run this command.
- In SurfaceBrowser, click on the Project tab located in the top right-hand corner of the screen. From here, you can either create a new project or select an existing one. For instance, you may choose to select the "D3 Security" project.
- Click on the project to view its details. To obtain the project ID, you can look at the URL of the project page and find the alphanumeric code following the "project/" segment of the URL.
Query is an optional parameter to run this command.
- You can use this parameter to perform partial searches on either the description (accessible in the raw data returned by this command at the path $.data[*].add_rules[*].description) or the name (accessible in the raw data returned by this command at the path $.data[*].add_rules[*].name).

Input

Input Parameter	Required/Optional	Description	Example
Project ID	Required	The ID of the project to retrieve risk issues. Project IDs can be obtained from the SurfaceBrowser Web user interface by navigating to the project section and selecting the desired project. The Project ID can be found in the URL of the project page.	Ebb*****b6
Start Time	Optional	The timestamp to get rule changes afterwards.	2023-03-01 00:00
End Time	Optional	The timestamp to get risk issue history up to.	2023-03-02 00:00
Rule Action	Optional	The rule action (i.e., added or removed) to filter rule changes. If this parameter is not defined, both added rules and removed rules will be returned.	Added Rules
Classification	Optional	The classification level to filter rule changes. If this parameter is not defined, issues of any classification level will be returned.	Informational
Query	Optional	The freetext search string to filter rule changes by name and description fields.	Hosts with Self-Signed SSL/TLS

Output

Raw Data

The primary response data from the API request.

SAMPLE DATA

JSON

{
  "data": [
      {
          "added_rules": [
              {
                  "classification": "informational",
                  "description": "1 hosts seen using a certificate where subject and issuer names are the same.",
                  "example_entities": {
                      "domains": [
                          {
                              "additional": "Issuer: Kubernetes Ingress Controller Fake Certificate",
                              "example": "https://*****.com/***",
                              "sort_value": 35
                          }
                      ],
                      "ips": []
                  },
                  "id": "***************************************",
                  "name": "Hosts with Self-Signed SSL/TLS Certificates",
                  "rule_metadata": {
                      "entity_counts": {
                          "domains": 1,
                          "ips": 0
                      },
                      "references": []
                  }
              }
          ],
          "counts": {
              "added_rules": 1,
              "removed_rules": 0
          },
          "previous_risk_score": 0,
          "previous_snapshot": null,
          "removed_rules": [],
          "risk_score": 26,
          "snapshot": "2022-12-15 21:44:03"
      }
  ],
  "meta": {
      "params": {
          "classification": [
              "high",
              "moderate",
              "informational"
          ],
          "end": "2022-12-15 21:44:03",
          "query": "Hosts with Self-Signed SSL/TLS",
          "rule_action": "added, removed",
          "start": "2022-12-15 21:44:03"
      }
  }
}

Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.
The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE

{
    "SnapshotTime": [
        "2022-12-15 21:44:03"
    ],
    "RiskScores": [
        26
    ],
    "AddedRulesCounts": [
        1
    ],
    "RemovedRulesCounts": [
        0
    ]
}

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

A connection issue with the integration
The API returned an error message
No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE

Successful

Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

params

{'classification': ['high', 'moderate', 'informational'], 'end': '2022-12-15 22:00:00', 'query': 'Hosts with Self-Signed SSL/TLS', 'rule_action': 'added, removed', 'start': '2022-12-15 21:00:00'}

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error	Description	Example
Failure Indicator	Indicates the command failure that happened at a specific input and/or API call.	Get Issue History failed.
Status Code	The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Recorded Future-SecurityTrails portal. Refer to the HTTP Status Code Registry for details.	Status Code: 404.
Message	The raw data or captured key error message from the integration API server about the API request failure.	Message: Project ID Not Found.

Error Sample Data

Get Issue History failed.

Status Code: 404.

Message: Project ID Not Found.

SQL Query

Runs a Structured Query Language (SQL) query on SecurityTrails data. For more information, such as properties and operators, see SQL Reference.

Input

Input Parameter

Required/Optional

Description

Example

Query

Required

The SQL query statement to run the query. The structure of the query statement is the following: SELECT attribute FROM table WHERE condition = "value". The query can be used to retrieve information from two tables: the hosts table and the IPs table. The hosts table contains information related to domains, hostnames, and other related hostname details, while the IPs table contains information related to IP addresses, ASNs, and other IP-related details.

To avoid API plan overages, it is recommended to narrow down your query condition by defining more specific conditions and values. For more information, such as properties and operators, see SQL Reference.

SELECT domain.hostname, dns.a.value.ip,ip.geo.owner.country FROM hosts WHERE domain.apex = 'google.com' AND dns.a.value.ip neq Null AND ip.geo.owner.country_iso_code = 'CN'

Output

Raw Data

The primary response data from the API request.

SAMPLE DATA

JSON

{
  "endpoint": "/v1/query/scroll",
  "id": "***************************************",
  "query": "SELECT domain.hostname, dns.a.value.ip,ip.geo.owner.country FROM hosts WHERE domain.apex = 'https://*****.com/***' AND dns.a.value.ip neq Null AND ip.geo.owner.country_iso_code = 'CN'",
  "records": [
      {
          "dns": {
              "a": {
                  "value": {
                      "ip": [
                          "***.***.***.***"
                      ]
                  }
              }
          },
          "domain": {
              "hostname": "https://*****.com/***"
          },
          "ip": {
              "geo": {
                  "owner": {
                      "country": [
                          "China"
                      ]
                  }
              }
          }
      },
      {
          "dns": {
              "a": {
                  "value": {
                      "ip": [
                          "***.***.***.***"
                      ]
                  }
              }
          },
          "domain": {
              "hostname": "https://*****.com/***"
          },
          "ip": {
              "geo": {
                  "owner": {
                      "country": [
                          "China"
                      ]
                  }
              }
          }
      }
  ],
  "total": {
      "relation": "eq",
      "value": 943
  }
}

Key Fields

Common cyber security indicators such as unique IDs, file hash values, CVE numbers, IP addresses, etc., will be extracted from Raw Data as Key Fields.
The system stores these key fields in the path $.[playbookTask].outputData. You can use these key-value pairs as data points for playbook task inputs.

SAMPLE DATA

CODE

{
    "ObtainedVsTotal": "eq",
    "TotalResultCounts": 943
}

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

A connection issue with the integration
The API returned an error message
No response from the API

You can view more details about an error in the Error tab.

Return Data can be passed down directly to a subsequent command or used to create conditional tasks in playbooks.

SAMPLE DATA

CODE

Successful

Result

Provides a brief summary of outputs in an HTML formatted table.

SAMPLE DATA

relation	eq
value	943

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error	Description	Example
Failure Indicator	Indicates the command failure that happened at a specific input and/or API call.	SQL Query failed.
Status Code	The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Recorded Future-SecurityTrails portal. Refer to the HTTP Status Code Registry for details.	Status Code: 403.
Message	The raw data or captured key error message from the integration API server about the API request failure.	Message: Invalid Query.

Error Sample Data

SQL Query failed.

Status Code: 403.

Message: Invalid Query.

Test Connection

Allows you to perform a health check on an integration connection. You can schedule a periodic health check by selecting Connection Health Check when editing an integration connection.

Input

N/A

Output

Return Data

Indicates one of the possible command execution states: Successful or Failed.

The Failed state can be triggered by any of the following errors:

A connection issue with the integration

The API returned an error message

No response from the API

You can view more details about an error in the Error tab.

SAMPLE DATA

CODE

Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the responses from the third-party API calls including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Parts in Error	Description	Example
Failure Indicator	Indicates the command failure that happened at a specific input and/or API call.	Test Connection failed. Failed to check the connector.
Status Code	The response code issued by the third-party API server or the D3 SOAR system that can be used to locate the corresponding error category. For example, if the returned status code is 401, the selected connection is unauthorized to run the command. The user or system support would need to check the permission setting in the Recorded Future-SecurityTrails portal. Refer to the HTTP Status Code Registry for details.	Status Code: 403.
Message	The raw data or captured key error message from the integration API server about the API request failure.	Message: Invalid authentication credentials.

Error Sample Data

Test Connection failed. Failed to check the connector.

Status Code: 403.

Message: Invalid authentication credentials.