Purging SQL Server Data

LAST UPDATED: MAY 21, 2025

Managing disk space on the Microsoft SQL Server is critical to maintaining a stable and functional D3 SOAR deployment. To prevent service interruptions caused by full disk volumes in on-premises environments, administrators can run a stored procedure to purge obsolete SQL data.

EXAMPLE

SQL

EXEC spSys_PurgeCyberData
    @StartDate                    = '2023-04-18 00:50:00',
    @EndDate                      = '2023-04-26 10:00:00',
    -- OR use this instead:
    -- @RetainDays                = 60,

    @PurgeIncidentsWithEvents     = 1,
    @PurgeEventFetchLog           = 1,
    @PurgeWebhookLog              = 1,
    @PurgeFileLog                 = 1,
    @PurgeCommandScheduleLog      = 1,
    @PurgeEventBatchLog           = 1,
    @PurgeJobQueue                = 1,
    @PurgePlaybookRuntimeData     = 1,
    @PurgeAllLogs                 = 1,
    @SiteIds                      = '1,2';

Parameters

Purge Eligibility Parameters

These parameters define the time window that determines which records are eligible for purging. Only one method—either @RetainDays or the combination of @StartDate and @EndDate—may be used in a single execution.

Parameter	Description
@StartDate	The date-time value in UTC (format: YYYY-MM-DD HH:MM:SS) used to select data for purging.
@EndDate	The date-time value in UTC (format: YYYY-MM-DD HH:MM:SS) marking the upper boundary for data to purge.
@RetainDays	The integer number of most recent days of data to retain, counting back from the time the event is ingested, in UTC.

Purge Control Parameters

These parameters specify the types of data to purge. Most parameters must be explicitly set to 1 to take effect, with the exception of @SiteIds, an nvarchar(max) string used to scope the purge to specific sites.

Parameter	Description
@ClosedIncidentOnly	Purges only incidents with status set to Closed.
@DismissedEventOnly	Purges only events marked as Dismissed.
@MaxRecordCount	The maximum number of latest records (events/incidents) to retain. Not site-specific.
@PurgeAllLogs	Purges internal system logs.
@PurgeAllTempTables	Purges internal system logs.
@PurgeCommandScheduleLog	Purges logs for scheduled command executions. These logs can be viewed in the Configuration > Schedules module.
@PurgeEventBatchLog	Purges runtime data for preprocessing playbooks. Runtime data can be viewed in the Preprocessing Playbook Viewer module.
@PurgeEventBatchLogKeepPlaybookName NEW	Retains preprocessing playbook names and purges all runtime data, leaving only the root node visible. Runtime data can be viewed in Preprocessing Playbook Viewer module.
@PurgeEventFetchLog	Purges logs generated by scheduled data ingestion via Fetch. These logs can be viewed in the Configuration > Data Ingestion module.
@PurgeFileLog	Purges logs associated with file uploads performed via the Investigation Dashboard > > Upload File menu option, as well as the uploaded files themselves. These logs can be viewed in the Configuration > Data Ingestion module.
@PurgeIncidentPlaybookRuntime	Purges runtime data of incident playbooks based on their attachment date. Runtime data can be viewed in the Investigation Dashboard > Incident Workspace > Playbooks module.
@PurgeIncidentPlaybookRuntimeKeepPlaybookName NEW	Retains investigation playbook names and purges all runtime data, leaving only the root node visible. Runtime data can be viewed in the Investigation Dashboard > Incident Workspace > Playbooks module.
@PurgeIncidentPlaybookRuntimeWithClosedIncidentOnly	Purges investigation playbook runtime data if the associated incident has a status of Closed. Runtime data can be viewed in the Investigation Dashboard > Incident Workspace > Playbooks module.
@PurgeIncidentsWithEvents	Purges events if all their linked incidents are also purged. Events that fall within the purge-eligible window will be skipped if they are linked to incidents that are not purge-eligible.
@PurgeJobQueue	Purges metadata history associated with the playbook task execution by the playbook engine.
@PurgeOthers	Purges internal system logs.
@PurgePlaybookRuntimeData	Purges all playbook runtimes (incident and event), functionally combining the effects of `@PurgeEventBatchLog` and `@PurgeIncidentPlaybookRuntime`. Runtime data can be viewed in the Preprocessing Playbook Viewer module.
@PurgeWebhookLog	Purges logs from scheduled data ingestion via webhook. These logs can be viewed in the Configuration > Data Ingestion module.
@PurgeClosedIncidentPlaybookRuntimeExcludeReopenTrigger	Purges runtime data for closed incident playbooks that do not have a reopen trigger rendered on the root node. Runtime data can be viewed in the Investigation Dashboard > Incident Workspace > Playbooks module.
@SiteIds	The IDs of the sites to which event purging should be applied when @PurgeIncidentsWithEvents equals 1. This parameter is relevant only when used in conjunction with @PurgeIncidentsWithEvents.

FAQs

Does a client need to manage disk space in the SaaS version of D3 SOAR?

D3 Security proactively monitors disk usage for SaaS environments and takes the following measures to ensure uninterrupted service:

Internal Alerting – Automated alerts notify internal teams before disk capacity becomes critical.
Scheduled Data Retention:
- Events and Incidents – Automatically purged every 13 months.
- Logs – Automatically purged every 3 months.
Client Coordination – D3 initiates conversations with affected clients before proceeding with any purge activities.

What is a stored procedure, and how does it relate to a SQL command?

Stored procedure – A predefined set of SQL operations stored in the database under a unique name. It is like a function defined and stored in the database.

In the example above, spSys_PurgeCyberData is the stored procedure.

SQL command – The statement used to execute a stored procedure.

In the example above, EXEC spSys_PurgeCyberData is the SQL command.

What are parameters, and why are they important when executing a stored procedure?

Parameters are named placeholders that serve as inputs that the stored procedure uses to determine how to execute its logic under a given context.
Adjusting parameters changes how a stored procedure behaves but does not alter its internal logic.
Without parameters, the procedure would have to be rewritten or duplicated for every variation in purge behavior.