Recorded Future-SecurityTrails

LAST UPDATED: OCT 31, 2024

Overview

SecurityTrails is a total inventory that curates comprehensive domain and IP address data for users and applications that demand clarity. By combining current and historic data of all Internet assets, SecurityTrails is the proven solution for 3rd-party risk assessment, attack surface reduction and threat hunting. This integration enables organizations to query risk issues and correlate domain and IP address data for risk assessment and threat hunting.

D3 SOAR is providing REST operations to function with Recorded Future-SecurityTrails.

Recorded Future-SecurityTrails is available for use in:

Known Limitations

It is important to monitor your account quota, as exceeding your credit limit will prevent commands from running. You can edit your subscription to purchase additional credits if you anticipate needing a higher quota.

Connection

To connect to Recorded Future-SecurityTrails from D3 SOAR, please follow this part to collect the required information below:

Permission Requirements

No permissions are required to create API keys in SecurityTrails.

Configuring Recorded Future-SecurityTrails to Work with D3 SOAR

1. Log in to your SecurityTrails account using your username and password.

   

2. Locate the API section on the left navigation menu and click on API Keys.

   

3. Next, select Create new API Key and enter a name for the key. Click Create new API Key to proceed.

   

4. Your new API key will now be visible in the list of API keys. Store this key securely to configure an integration connection with D3 SOAR.

   

Configuring D3 SOAR to Work with Recorded Future-SecurityTrails

1. Log in to D3 SOAR.

2. Find the Recorded Future-SecurityTrails integration.

   

   1. Navigate to Configuration on the top header menu.

   2. Click on the Integration icon on the left sidebar.

   3. Type Recorded Future-SecurityTrails in the search box to find the integration, then click it to select it.

   4. Click + New Connection, on the right side of the Connections section. A new connection window will appear.

3. Configure the following fields to create a connection to Recorded Future-SecurityTrails.

   

   1. Connection Name: The desired name for the connection.

   2. Site: Specifies the site to use the integration connection. Use the drop-down menu to select the site. The Share to Internal Sites option enables all sites defined as internal sites to use the connection. Selecting a specific site will only enable that site to use the connection.

   3. Recipient site for events from connections Shared to Internal Sites: This field appears if you selected Share to Internal Sites for Site to let you select the internal site to deploy the integration connection.

   4. Agent Name (Optional): Specifies the proxy agent required to build the connection. Use the dropdown menu to select the proxy agent from a list of previously configured proxy agents.

   5. Description (Optional): Add your desired description for the connection.

   6. Configure User Permissions: Defines which users have access to the connection.

   7. Active: Check the tick box to ensure the connection is available for use.

      

   8. System: This section contains the parameters defined specifically for the integration. These parameters must be configured to create the integration connection.
      1. Input your domain level Server URL. The default value is https://api.securitytrails.com.
      2. Input your API Key. See Step 4 of Configuring Recorded Future-SecurityTrails to Work with D3 SOAR for instructions on obtaining the API Key.
      3. Input your API Version. The default value is v1.

   9. Connection Health Check: Updates the connection status you have created. A connection health check is done by scheduling the Test Connection command of this integration. This can only be done when the connection is active.
      To set up a connection health check, check the Connection Health Check tickbox. You can customize the interval (minutes) for scheduling the health check. An email notification can be set up after a specified number of failed connection attempts.

   10. Enable Password Vault: An optional feature that allows users to take the stored credentials from their own password vault. Please refer to the password vault connection guide if needed.

4. Test the connection.

   

   1. Click Test Connection to verify the account credentials and network connection. If the Test Connection Passed alert window appears, the test connection is successful. You will see Passed with a green checkmark appear beside the Test Connection button. If the test connection fails, please check your connection parameters and try again.

   2. Click OK to close the alert window.

   3. Click + Add to create and add the configured connection.

Commands

Recorded Future-SecurityTrails includes the following executable commands for users to set up schedules or create playbook workflows. With the Test Command, you can execute these commands independently for playbook troubleshooting.

Integration API Note

For more information about the Recorded Future-SecurityTrails API, please refer to the following SecurityTrails API references:

• How to use the SQL API

• Details

DSL Query

Runs a Domain Specific Language (DSL) query on the SecurityTrails Exploration End Point platform. This command lets you query SecurityTrails Exploration End Point with flexible SQL-like queries. The syntax used for SecurityTrails' DSL closely resembles that of SQL WHERE predicates.

Input

Output

{
    "endpoint": "/v1/search/list",
    "meta": {
        "max_page": 1,
        "page": 1,
        "query": "ns = 'https://*****.com/***' AND ipv4 = '***.***.***.***'",
        "total_pages": 1
    },
    "record_count": 2,
    "records": [
        {
            "alexa_rank": 10,
            "computed": {
                "company_name": "Yahoo! Inc."
            },
            "host_provider": [
                "Oath Holdings Inc."
            ],
            "hostname": "https://*****.com/***",
            "ips": [
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***"
            ],
            "mail_provider": [
                "Oath Holdings Inc."
            ],
            "whois": {
                "createdDate": 790416000,
                "expiresDate": 1674104400,
                "registrar": "MarkMonitor, Inc."
            }
        },
        {
            "alexa_rank": null,
            "computed": {
                "company_name": "Domains By Proxy, LLC"
            },
            "host_provider": [
                "Oath Holdings Inc."
            ],
            "hostname": "https://*****.com/***",
            "ips": [
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***",
                "***.***.***.***"
            ],
            "mail_provider": [
                "Oath Holdings Inc."
            ],
            "whois": {
                "createdDate": 1581610540,
                "expiresDate": 1676304940,
                "registrar": "Alibaba Cloud Computing (Beijing) Co., Ltd."
            }
        }
    ]
}

{
    "RecordCount": 2
}

Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Fetch Incident

Returns a list of hosts with issue updates.

Input

Output

Successful

{
    "Results": [
        {
            "HostType": "domain",
            "HostValue": "reach.d3security.com",
            "snapshot": "2022-12-15 21:44:03",
            "Issues": [
                {
                    "classification": "moderate",
                    "description": "1 hosts seen using a certificate where subject and issuer names are the same.",
                    "example_entities": {
                        "domains": [
                            {
                                "additional": "Issuer: Kubernetes Ingress Controller Fake Certificate",
                                "example": "reach.d3security.com",
                                "sort_value": 35
                            }
                        ]
                    },
                    "id": "crawl_self-signed-certificate",
                    "name": "Hosts with Self-Signed SSL/TLS Certificates",
                    "rule_metadata": {
                        "entity_counts": {
                            "domains": 1,
                            "ips": 0
                        },
                        "references": [],
                        "supports_evidence": false
                    },
                    "Status": "Added",
                    "snapshot": "2022-12-15 21:44:03"
                }
            ]
        }
    ],
    "D3Errors": []
}

{
  "HostValues": ["reach.d3security.com"],
  "HostTypes": ["domain"]
}

Incident Field Mapping

For this integration, the default incident fields in D3 SOAR are fixed with no built-in source fields. Users can specify the source fields as needed. 

Event and Incident Intake Field Mapping

Please note that incident and event intake commands require both Event Field and Incident Field Mapping. These field mappings are the default event/incident field mappings for D3 system integrations. You can edit the provided mappings or create custom mappings as needed. Please refer to Event and Incident Intake Field Mapping for more details.

Incident Main JSON Path: $.Results

Event Field Mapping

Fetch Event commands require event field mapping. Field mapping plays a key role in the data normalization process part of the event pipeline. Field mapping converts the original data fields from the different providers to the D3 fields which are standardized by the D3 Model. Please refer to Event and Incident Intake Field Mapping for more details.

To custom field mapping, click + Add Field and add the custom field of your choice. You can also remove built-in field mappings by clicking x. Please note that two underscore characters will automatically prefix the defined Field Name as the System Name for a custom field mapping. Additionally, if an input Field Name contains any spaces, they will automatically be replaced with underscores for the corresponding System Name.

As a system integration, the Recorded Future-SecurityTrails integration has some pre-configured field mappings for default field mapping.

• Default Event Source
  The Default Event Source is the default set of field mappings that are applied when this fetch event command is executed. For out-of-the-box integrations, you will find a set of field mapping provided by the system. Default event source provides field mappings for common fields from fetched Issues. The default event source has a “Main Event JSON Path” (i.e., $.Issues) that is used to extract a batch of events from the response raw data. Click Edit Main JSON Path to view the “Main Event JSON Path”.

  

• Main Event JSON Path: $.Issues

The Main Event JSON Path determines the root path where the system starts parsing raw response data into D3 event data. The JSON path begins with $, representing the root element. The path is formed by appending a sequence of child elements to $, each separated by a dot (.). Square brackets with nested quotation marks ([‘...’]) should be used to separate child elements in JSON arrays.

For example, the root node of a JSON Path is Issues. The child node denoting the Event code field would be id. Putting it together, the JSON Path expression to extract the Event code is $.Issues.id.

The pre-configured field mappings are detailed below:

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Get Current Issues

Retrieves current risk issues of the specified project.

READER NOTE

Project ID is a required parameter to run this command.

• In SurfaceBrowser, click on the Project tab located in the top right-hand corner of the screen. From here, you can either create a new project or select an existing one. For instance, you may choose to select the "D3 Security" project.

  

• Click on the project to view its details. To obtain the project ID, you can look at the URL of the project page and find the alphanumeric code following the "project/" segment of the URL.

  

Input

Output

{
  "data": [
      {
          "classification": "informational",
          "description": "2 hosts seen supporting Basic Authentication",
          "example_entities": {
              "domains": [
                  {
                      "additional": "Basic realm=\"d3secstage\"",
                      "example": "https://*****.com/***",
                      "sort_value": 60
                  },
                  {
                      "additional": "Basic realm=\"d3secdev\"",
                      "example": "https://*****.com/***",
                      "sort_value": 60
                  }
              ]
          },
          "id": "***************************************",
          "name": "Hosts with Basic Authentication",
          "rule_metadata": {
              "entity_counts": {
                  "domains": 2,
                  "ips": 0
              },
              "references": []
          }
      },
      {
          "classification": "informational",
          "description": "2 potential development servers are addressable on the public internet",
          "example_entities": {
              "domains": [
                  {
                      "additional": "Staging",
                      "example": "https://*****.com/***",
                      "sort_value": 35
                  },
                  {
                      "additional": "Development",
                      "example": "https://*****.com/***",
                      "sort_value": 35
                  }
              ]
          },
          "id": "***************************************",
          "name": "Development Infrastructure Exposure",
          "rule_metadata": {
              "entity_counts": {
                  "domains": 2,
                  "ips": 0
              },
              "references": []
          }
      },
      {
          "classification": "informational",
          "description": "1 hosts seen using a certificate where subject and issuer names are the same.",
          "example_entities": {
              "domains": [
                  {
                      "additional": "Issuer: Kubernetes Ingress Controller Fake Certificate",
                      "example": "https://*****.com/***",
                      "sort_value": 35
                  }
              ]
          },
          "id": "***************************************",
          "name": "Hosts with Self-Signed SSL/TLS Certificates",
          "rule_metadata": {
              "entity_counts": {
                  "domains": 1,
                  "ips": 0
              },
              "references": []
          }
      }
  ],
  "meta": {
      "counts": {
          "informational": 3
      },
      "project_id": "***-***-***-***-***",
      "project_title": "D3 Security",
      "snapshot": "2022-12-15 21:44:03"
  }
}

{
    "Names": [
        "Hosts with Basic Authentication"
    ],
    "IDs": [
        "crawl_basic-authentication"
    ],
    "Classifications": [
        "informational"
    ],
    "Descriptions": [
        "2 hosts seen supporting Basic Authentication"
    ],
    "DomainCounts": [
        2
    ],
    "IPCounts": [
        0
    ]
}

Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Get Issue History

Retrieves a risk issue's history based on the specified criteria.

READER NOTE

• Project ID is a required parameter to run this command.

  • In SurfaceBrowser, click on the Project tab located in the top right-hand corner of the screen. From here, you can either create a new project or select an existing one. For instance, you may choose to select the "D3 Security" project.

    

  • Click on the project to view its details. To obtain the project ID, you can look at the URL of the project page and find the alphanumeric code following the "project/" segment of the URL.

    

• Query is an optional parameter to run this command.

  • You can use this parameter to perform partial searches on either the description (accessible in the raw data returned by this command at the path $.data[*].add_rules[*].description) or the name (accessible in the raw data returned by this command at the path $.data[*].add_rules[*].name).

Input

Output

{
  "data": [
      {
          "added_rules": [
              {
                  "classification": "informational",
                  "description": "1 hosts seen using a certificate where subject and issuer names are the same.",
                  "example_entities": {
                      "domains": [
                          {
                              "additional": "Issuer: Kubernetes Ingress Controller Fake Certificate",
                              "example": "https://*****.com/***",
                              "sort_value": 35
                          }
                      ],
                      "ips": []
                  },
                  "id": "***************************************",
                  "name": "Hosts with Self-Signed SSL/TLS Certificates",
                  "rule_metadata": {
                      "entity_counts": {
                          "domains": 1,
                          "ips": 0
                      },
                      "references": []
                  }
              }
          ],
          "counts": {
              "added_rules": 1,
              "removed_rules": 0
          },
          "previous_risk_score": 0,
          "previous_snapshot": null,
          "removed_rules": [],
          "risk_score": 26,
          "snapshot": "2022-12-15 21:44:03"
      }
  ],
  "meta": {
      "params": {
          "classification": [
              "high",
              "moderate",
              "informational"
          ],
          "end": "2022-12-15 21:44:03",
          "query": "Hosts with Self-Signed SSL/TLS",
          "rule_action": "added, removed",
          "start": "2022-12-15 21:44:03"
      }
  }
}

{
    "SnapshotTime": [
        "2022-12-15 21:44:03"
    ],
    "RiskScores": [
        26
    ],
    "AddedRulesCounts": [
        1
    ],
    "RemovedRulesCounts": [
        0
    ]
}

Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

SQL Query

Runs a Structured Query Language (SQL) query on SecurityTrails data. For more information, such as properties and operators, see SQL Reference.

Input

Output

{
  "endpoint": "/v1/query/scroll",
  "id": "***************************************",
  "query": "SELECT domain.hostname, dns.a.value.ip,ip.geo.owner.country FROM hosts WHERE domain.apex = 'https://*****.com/***' AND dns.a.value.ip neq Null AND ip.geo.owner.country_iso_code = 'CN'",
  "records": [
      {
          "dns": {
              "a": {
                  "value": {
                      "ip": [
                          "***.***.***.***"
                      ]
                  }
              }
          },
          "domain": {
              "hostname": "https://*****.com/***"
          },
          "ip": {
              "geo": {
                  "owner": {
                      "country": [
                          "China"
                      ]
                  }
              }
          }
      },
      {
          "dns": {
              "a": {
                  "value": {
                      "ip": [
                          "***.***.***.***"
                      ]
                  }
              }
          },
          "domain": {
              "hostname": "https://*****.com/***"
          },
          "ip": {
              "geo": {
                  "owner": {
                      "country": [
                          "China"
                      ]
                  }
              }
          }
      }
  ],
  "total": {
      "relation": "eq",
      "value": 943
  }
}

{
    "ObtainedVsTotal": "eq",
    "TotalResultCounts": 943
}

Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the details responded from D3 SOAR or third-party API calls, including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.

Test Connection

Allows you to perform a health check on an integration connection. You can schedule a periodic health check by selecting Connection Health Check when editing an integration connection.

Input

N/A

Output

Successful

Error Handling

If the Return Data is Failed, an Error tab will appear in the Test Result window.

The error tab contains the responses from the third-party API calls including Failure Indicator, Status Code, and Message. This can help you locate the root cause of a command failure.