diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml index eb083825f63..00f01b6389d 100644 --- a/config/_default/menus/main.en.yaml +++ b/config/_default/menus/main.en.yaml @@ -7060,9 +7060,9 @@ menu: identifier: cloud_siem_custom_detection_rules weight: 202 - name: Create Rule - url: security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/real_time_rule + url: security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule parent: cloud_siem_custom_detection_rules - identifier: cloud_siem_real_time_rule + identifier: cloud_siem_create_custom_detection_rule weight: 2021 - name: Anomaly url: security/cloud_siem/detect_and_monitor/custom_detection_rules/anomaly diff --git a/content/.gitignore b/content/.gitignore index b5bf2e9ed95..89b0c0c57d7 100644 --- a/content/.gitignore +++ b/content/.gitignore @@ -31,3 +31,4 @@ /en/synthetics/notifications/template_variables/mobile.md /en/synthetics/notifications/template_variables/multistep.md /en/synthetics/notifications/template_variables/api.md +/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.md diff --git a/content/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.md b/content/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.md deleted file mode 100644 index 139ec6d3919..00000000000 --- a/content/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Create a Custom Rule -type: multi-code-lang ---- - diff --git a/content/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.mdoc.md b/content/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.mdoc.md new file mode 100644 index 00000000000..9bc6abdca95 --- /dev/null +++ b/content/en/security/cloud_siem/detect_and_monitor/custom_detection_rules/create_rule/_index.mdoc.md @@ -0,0 +1,1017 @@ +--- +title: Create a Custom Rule +content_filters: + - trait_id: cloud_siem_detection_rule_type + option_group_id: cloud_siem_detection_rule_type_options + - trait_id: cloud_siem_detection_rule_search_query + option_group_id: cloud_siem_detection__search_query_options +--- + + + +## Overview + +{% if equals($cloud_siem_detection_rule_type, "real_time_rule") %} +Real-time detection rules continuously monitors and analyzes incoming logs for security threats. These rules trigger immediate alerts when specific patterns or anomalies are detected, enabling quicker response to potential incidents. +{% /if %} +{% if equals($cloud_siem_detection_rule_type, "scheduled_rule") %} +Scheduled detection rules run at predefined intervals to analyze indexed log data and detect security threats. These rules can identify patterns, anomalies, or specific conditions within a defined time frame, and trigger alerts or reports if the criteria are met. + +Scheduled rules complement real-time monitoring by ensuring periodic, in-depth analysis of logs using [calculated fields][7]. +{% /if %} +{% if equals($cloud_siem_detection_rule_type, "historical_job") %} +Historical jobs are one-time executable queries on historical logs used to backtest detection rules and assess their effectiveness on past data. The generated job results are lightweight versions of signals providing information on potential threats and anomalies on historical logs. After reviewing the results, you can convert results needing immediate action into signals. +{% /if %} + +## Create a rule + +1. To create a detection rule, navigate to the [Create a New Detection][2] page. +1. {% if equals($cloud_siem_detection_rule_type, "real_time_rule") %}Select **Real-Time Rule**.{% /if %} +{% if equals($cloud_siem_detection_rule_type, "scheduled_rule") %}Select **Scheduled Rule**.{% /if %} +{% if equals($cloud_siem_detection_rule_type, "historical_job") %}Select **Historical job**, then select the **Logs Index** and **Timerange** for the job.{% /if %} +1. {% if equals($cloud_siem_detection_rule_search_query, "threshold") %}Select the **Threshold** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "new_value") %}Select the **New value** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "anomaly") %}Select the **Anomaly** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "content_anomaly") %}Select the **Content Anomaly** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "impossible_travel") %}Select the **Impossible travel** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "third_party") %}Select the **Third party** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "sequence") %}Select the **Sequence** detection method.{% /if %} +{% if equals($cloud_siem_detection_rule_search_query, "signal_correlation") %}Select the **Signal correlation** detection method.{% /if %} + +## Define your search query + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "threshold")) %} + +{% img src="security/security_monitoring/detection_rules/threshold_20250310.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/threshold_query.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "new_value")) %} +{% img src="security/security_monitoring/detection_rules/new_value_20250310.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/new_value_query.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "anomaly")) %} +{% img src="security/security_monitoring/detection_rules/anomaly_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. +1. (Optional) In the **Count** dropdown menu, select attributes whose unique values you want to count during the specified time frame. + {% partial file="security/cloud_siem/anomaly_query.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "content_anomaly")) %} +{% img src="security/security_monitoring/detection_rules/content_anomaly_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/content_anomaly_query.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "impossible_travel")) %} +{% img src="security/security_monitoring/detection_rules/impossible_travel_query.png" alt="Define the search query" style="width:100%;" /%} +{% alert level="info" %} +All logs and events matching this query are analyzed for potential impossible travel. +{% /alert %} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/impossible_travel_query.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "third_party")) %} +{% img src="security/security_monitoring/detection_rules/third_party_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a root query for your logs or events using the [Log Explorer search syntax][1]. +1. In the **Trigger for each new** dropdown menu, select the attributes where each attribute generates a signal for each new attribute value over a 24-hour roll-up period. +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Add Root Query** and repeat steps 2-4 to add and test additional queries. +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "sequence")) %} +{% img src="security/security_monitoring/detection_rules/sequence/sequence_queries.png" alt="Sequence editor page showing the sequence with two steps" style="width:100%;" /%} + +### Add step + +1. To search a different data type, click the down arrow next to **Logs** and select **Signals** or **Rules**. +1. Define the condition for the step. + - **Logs**: Construct a search query using the [Log Explorer search syntax][1]. + - **Signals**: Reference an existing rule or query on signal fields. + - **Rules**: Select a rule. +1. Set **group by** fields (for example, `@usr.email` or `@ip.address`) to link entities across steps. +1. Enter a threshold condition, such as `>10`. +1. If you want to use another query, connect this query with the next query using `AND` or `OR` and repeat steps 1-4. +1. In the **roll-up over** dropdown menu, select the time frame all queries in that step must occur to transition to the next step. + +### Define step transitions + +For the current step and the next step: + +1. In the **within** dropdown menu, select an evaluation window for the transition. + {% alert level="info" %} + The total evaluation time across the sequence can be up to 24 hours. + {% /alert %} +1. Follow the instructions in [Add step](#add-step) to complete the step. + {% alert level="info" %} + You can select different `group by` fields between steps. For example, link `@usr.email`from an earlier step to `@ip.address` in a later step. + {% /alert %} +1. Click **Add Step** if you want to add more steps. + +### Severity and notification + +1. In the **Trigger** dropdown menu, select the severity status. +1. (Optional) In the **And notify** section, click **Add Recipient** to configure [notification targets][3]. + - You can create [notification rules][4] to manage notifications automatically, avoiding manual edits for each detection rule. + +### Review the sequence preview + +1. In the **Preview detection** section, check the steps, transitions, and time window in the visualization of the steps. Reorder the steps and adjust time windows as needed. +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "signal_correlation")) %} +{% img src="security/security_monitoring/detection_rules/signal_correlation_query.png" alt="Define the search query" style="width:100%;" /%} + +1. Select a rule for **Rule a**. +1. Click the pencil icon to rename the rule. +1. Use the **correlated by** dropdown to define the correlating attribute. + - You can select multiple attributes (maximum of 3) to correlate the selected rules. +1. Select a rule for **Rule b** in the second Rule editor's dropdown. + - The attributes and sliding window time frame is automatically set to what was selected for **Rule a**. +1. Click the pencil icon to rename the rule. +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "threshold")) %} +Choose the query language you want to use. + +{% collapse-content title="Event Query" level="h4" expanded=false id="threshold-event-query" %} +{% img src="security/security_monitoring/detection_rules/threshold_20250310.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. If you are using an add-on and see the **Index** dropdown menu, select the index of logs you want to analyze. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/threshold_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /collapse-content %} +{% collapse-content title="SQL" level="h4" expanded=false id="threshold-sql" %} +You can use SQL syntax to write detection rules for additional flexibility, consistency, and portability. For information on the available syntax, see [DDSQL Reference][5]. + +In Datadog, SQL queries are compatible with data stored in [datasets][6]. You can create datasets to format data already stored in tables for the following data types: +- Logs +- Audit Trail logs +- Events +- Security signals +- Spans +- RUM events +- Product Analytics events +- Cloud Network data +- NetFlow data +- Reference tables +- Infrastructure tables + +{% img src="security/security_monitoring/detection_rules/sql-query-example.png" alt="Example of a SQL dataset and query" style="width:100%;" /%} + +1. Under **Define Datasets**, choose one or more datasets to use in your query. In the dropdown, you can select an existing published dataset to either use or clone, or click the **New** icon to create a database from scratch. + - If you chose an existing dataset and made changes, click **Update** to apply those changes to that dataset, or **Clone With Changes** to create a dataset with your changes applied. + - If you created a dataset, click **Create** so you can use it in your rule. +2. Under **Write Queries**, enter one or more SQL queries. For more information, see [DDSQL Reference][5]. Click **Preview** to see a list of matching results. + +{% /collapse-content %} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "new_value")) %} +{% img src="security/security_monitoring/detection_rules/new_value_20250310.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. If you are using an add-on and see the **Index** dropdown menu, select the index of logs you want to analyze. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/new_value_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "anomaly")) %} +{% img src="security/security_monitoring/detection_rules/anomaly_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. If you are using an add-on and see the **Index** dropdown menu, select the index of logs you want to analyze. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/anomaly_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "content_anomaly")) %} +{% img src="security/security_monitoring/detection_rules/content_anomaly_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. If you are using an add-on and see the **Index** dropdown menu, select the index of logs you want to analyze. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/content_anomaly_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "impossible_travel")) %} +{% img src="security/security_monitoring/detection_rules/impossible_travel_query.png" alt="Define the search query" style="width:100%;" /%} +{% alert level="info" %} +All logs and events matching this query are analyzed for potential impossible travel. +{% /alert %} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. If you are using an add-on and see the **Index** dropdown menu, select the index of logs you want to analyze. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/impossible_travel_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "third_party")) %} +{% img src="security/security_monitoring/detection_rules/third_party_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. If you are using an add-on and see the **Index** dropdown menu, select the index of logs you want to analyze. +1. Construct a root query for your logs or events using the [Log Explorer search syntax][1]. +1. In the **Trigger for each new** dropdown menu, select the attributes where each attribute generates a signal for each new attribute value over a 24-hour roll-up period. +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Add Root Query** and repeat steps 3-7 to add and test additional queries. +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "signal_correlation")) %} +{% img src="security/security_monitoring/detection_rules/signal_correlation_query.png" alt="Define the search query" style="width:100%;" /%} + +1. Select a rule for **Rule a**. +1. Click the pencil icon to rename the rule. +1. Use the **correlated by** dropdown to define the correlating attribute. + - You can select multiple attributes (maximum of 3) to correlate the selected rules. +1. Select a rule for **Rule b** in the second Rule editor's dropdown. + - The attributes and sliding window time frame is automatically set to what was selected for **Rule a**. +1. Click the pencil icon to rename the rule. +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "threshold")) %} +{% img src="security/security_monitoring/detection_rules/threshold_20250310.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/threshold_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "new_value")) %} +{% img src="security/security_monitoring/detection_rules/new_value_20250310.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/new_value_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "anomaly")) %} +{% img src="security/security_monitoring/detection_rules/anomaly_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/anomaly_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "content_anomaly")) %} +{% img src="security/security_monitoring/detection_rules/content_anomaly_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/content_anomaly_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "impossible_travel")) %} +{% img src="security/security_monitoring/detection_rules/impossible_travel_query.png" alt="Define the search query" style="width:100%;" /%} +{% alert level="info" %} +All logs and events matching this query are analyzed for potential impossible travel. +{% /alert %} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a search query for your logs or events using the [Log Explorer search syntax][1]. + {% partial file="security/cloud_siem/impossible_travel_query.mdoc.md" /%} +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Save Rule**. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "third_party")) %} +{% img src="security/security_monitoring/detection_rules/third_party_query.png" alt="Define the search query" style="width:100%;" /%} + +1. To search Audit Trail events or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. +1. Construct a root query for your logs or events using the [Log Explorer search syntax][1]. +1. In the **Trigger for each new** dropdown menu, select the attributes where each attribute generates a signal for each new attribute value over a 24-hour roll-up period. +1. (Optional) To create calculated fields that transform your logs during query time: + {% partial file="security/cloud_siem/add_calculated_fields.mdoc.md" /%} +1. (Optional) Filter logs using reference tables: + {% partial file="security/cloud_siem/add_reference_tables.mdoc.md" /%} +1. (Optional) To test your rules against sample logs, click **Unit Test**. + {% partial file="security/cloud_siem/unit_testing.mdoc.md" /%} +1. Click **Add Root Query** and repeat steps 2-6 to add and test additional queries. +1. Click **Save Rule**. +{% /if %} + +## Set conditions + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "threshold")) %} +{% img src="security/security_monitoring/detection_rules/condition_simple_then.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_threshold.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-rt-threshold %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-rt-threshold %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-threshold %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-threshold %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} + +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "new_value")) %} +{% img src="security/security_monitoring/detection_rules/severity_notification.png" alt="Set your severity and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_severity_notify_only.mdoc.md" /%} + +### Other parameters + +#### Forget value {% #forget-value-rt-new-value%} + +{% partial file="security/cloud_siem/forget_value.mdoc.md" /%} + +#### Rule multi-triggering behavior {% #rule-multi-triggering-rt-new-value%} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-new-value%} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-new-value%} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-new-value %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} + +#### Enable instantaneous baseline {% #enable-instantaneous-baseline-new-value %} + +{% partial file="security/cloud_siem/enable_instantaneous_baseline.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "anomaly")) %} +{% img src="security/security_monitoring/detection_rules/anomaly_notification.png" alt="Set your severity, anomaly percentile, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_anomaly.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-rt-anomaly %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-rt-anomaly %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "content_anomaly")) %} +{% img src="security/security_monitoring/detection_rules/condition_content_anomaly.png" alt="Set your condition, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_content_anomaly.mdoc.md" /%} + +### Other parameters + +#### Content anomaly detection {% #content-anomaly-rt-content-anomaly %} + +{% partial file="security/cloud_siem/content_anomaly_options.mdoc.md" /%} + +#### Rule multi-triggering behavior {% #rule-multi-triggering-rt-content-anomaly %} + +{% partial file="security/cloud_siem/rule_multi_triggering_content_anomaly.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-rt-content-anomaly %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-content-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-content-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "impossible_travel")) %} +{% img src="security/security_monitoring/detection_rules/severity_notification.png" alt="Set your severity and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_severity_notify_only.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-rt-impossible-travel %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-rt-impossible-travel %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-impossible-travel %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-impossible-travel %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "third_party")) %} +{% img src="security/security_monitoring/detection_rules/condition_else.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_third_party.mdoc.md" /%} + +### Other parameters + +#### Decrease severity for non-production environments {% #decrease-severity-rt-third-party %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-third-party %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-third-party %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "sequence")) %} +#### Rule multi-triggering {% #rule-multi-triggering-rt-sequence %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-rt-sequence %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-rt-sequence %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-rt-sequence %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_search_query, "signal_correlation")) %} +{% img src="security/security_monitoring/detection_rules/condition_simple_then.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_then_operator.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-rt-signal-correlation %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-rt-signal-correlation %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "threshold")) %} +{% img src="security/security_monitoring/detection_rules/condition_simple_then.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_threshold.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-scheduled-threshold %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-threshold %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-scheduled-threshold %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-threshold %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "new_value")) %} +{% img src="security/security_monitoring/detection_rules/severity_notification.png" alt="Set your severity and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_severity_notify_only.mdoc.md" /%} + +### Other parameters + +#### Forget value {% #forget-value-scheduled-new-value %} + +{% partial file="security/cloud_siem/forget_value.mdoc.md" /%} + +#### Rule multi-triggering behavior {% #rule-multi-triggering-scheduled-new-value %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-new-value %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-scheduled-new-value %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-new-value %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "anomaly")) %} +{% img src="security/security_monitoring/detection_rules/severity_notification.png" alt="Set your severity and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_severity_notify_only.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-scheduled-anomaly %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-anomaly %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-scheduled-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "content_anomaly")) %} +{% img src="security/security_monitoring/detection_rules/condition_content_anomaly.png" alt="Set your condition, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_content_anomaly.mdoc.md" /%} + +### Other parameters + +#### Content anomaly detection {% #content-anomaly-scheduled-content-anomaly %} + +{% partial file="security/cloud_siem/content_anomaly_options.mdoc.md" /%} + +#### Rule multi-triggering behavior {% #rule-multi-triggering-scheduled-content-anomaly %} + +{% partial file="security/cloud_siem/rule_multi_triggering_content_anomaly.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-content-anomaly %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-scheduled-content-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-content-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "impossible_travel")) %} +{% img src="security/security_monitoring/detection_rules/severity_notification.png" alt="Set your severity and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_severity_notify_only.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-scheduled-impossible-travel %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-impossible-travel %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-scheduled-impossible-travel %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-impossible-travel %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "third_party")) %} +{% img src="security/security_monitoring/detection_rules/condition_else.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_third_party.mdoc.md" /%} + +### Other parameters + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-third-party %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-scheduled-third-party %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-third-party %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "scheduled_rule"),equals($cloud_siem_detection_rule_search_query, "signal_correlation")) %} +{% img src="security/security_monitoring/detection_rules/condition_simple_then.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_then_operator.mdoc.md" /%} + +### Other parameters + +#### Rule multi-triggering {% #rule-multi-triggering-scheduled-signal-correlation %} + +{% partial file="security/cloud_siem/rule_multi_triggering.mdoc.md" /%} + +#### Decrease severity for non-production environments {% #decrease-severity-scheduled-signal-correlation %} + +{% partial file="security/cloud_siem/enable_decrease_severity.mdoc.md" /%} + +#### Group signals {% #group-signals-scheduled-signal-correlation %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "threshold")) %} +{% img src="security/security_monitoring/detection_rules/threshold_historical_condition.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +{% partial file="security/cloud_siem/set_conditions_threshold.mdoc.md" /%} + +### Other parameters + +#### Job multi-triggering {% #job-multi-triggering-threshold %} + +{% partial file="security/cloud_siem/job_multi_triggering.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-historical-threshold %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-historical-threshold %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "new_value")) %} +### Other parameters + +#### Forget value {% #forget-value-historical-new-value %} + +{% partial file="security/cloud_siem/forget_value.mdoc.md" /%} + +#### Job multi-triggering behavior {% #job-multi-triggering-historical-new-value %} + +{% partial file="security/cloud_siem/job_multi_triggering.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-historical-new-value %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-historical-new-value %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "anomaly")) %} +### Other parameters + +#### Job multi-triggering {% #job-multi-triggering-historical-anomaly %} + +{% partial file="security/cloud_siem/job_multi_triggering.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-historical-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-historical-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} + +#### Bucket duration + +In the **Bucket Duration** dropdown, select a duration over which to measure percentiles. + +#### Learning duration + +In the **Learning Duration** dropdown, select an amount of time for the rule to learn new values. + +#### Learning period alerts + +In the **Learning Period Alerts** dropdown, choose whether you want Cloud SIEM to send alerts during the learning period. +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "content_anomaly")) %} +{% img src="security/security_monitoring/detection_rules/content_anomaly_historical_condition.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +1. (Optional) Click the pencil icon next to **Condition 1** if you want to rename the condition. This name is appended to the rule name when a signal is generated. +1. In the **Anomaly count** field, enter the condition for how many anomalous logs within the specified window are required to trigger a signal. + - For example, if the condition is `a >= 3` where `a` is the query, a signal is triggered if there are at least three anomalous logs within the evaluation window. + - All rule conditions are evaluated as condition statements. Thus, the order of the conditions affects which notifications are sent because the first condition to match generates the signal. Click and drag your rule conditions to change their ordering. + - A rule condition contains logical operations (`>`, `>=`, `&&`, `||`) to determine if a signal should be generated based on the event counts in the previously defined queries. + - The ASCII lowercase query labels are referenced in this section. An example rule condition for query `a` is `a > 3`. + {% alert level="info" %} + The query label must precede the operator. For example, `a > 3` is allowed; `3 < a` is not allowed. + {% /alert %} +1. In the **within a window of** dropdown menu, select the time period during which a signal is triggered if the condition is met. + - An `evaluation window` is specified to match when at least one of the cases matches true. This is a sliding window and evaluates cases in real time. + +### Other parameters + +#### Content anomaly detection {% #content-anomaly-historical-content-anomaly %} + +{% partial file="security/cloud_siem/content_anomaly_options.mdoc.md" /%} + +#### Job multi-triggering behavior {% #job-multi-triggering-historical-content-anomaly %} + +{% partial file="security/cloud_siem/rule_multi_triggering_content_anomaly.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-historical-content-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-historical-content-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "impossible_travel")) %} +### Other parameters + +#### Job multi-triggering {% #job-multi-triggering-historical-anomaly %} + +{% partial file="security/cloud_siem/job_multi_triggering.mdoc.md" /%} + +#### Enable optional group by {% #enable-group-by-historical-anomaly %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-historical-anomaly %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if and(equals($cloud_siem_detection_rule_type, "historical_job"),equals($cloud_siem_detection_rule_search_query, "third_party")) %} +{% img src="security/security_monitoring/detection_rules/set_condition_root_query.png" alt="Set your conditions, severity, and notification recipients" style="width:100%;" /%} + +1. (Optional) Click the pencil icon next to **Condition 1** if you want to rename the condition. This name is appended to the rule name when a signal is generated. +1. In the **Query** field, enter the tags of a log that you want to trigger a signal. + - For example, if you want logs with the tag `dev:demo` to trigger signals with a severity of `INFO`, enter `dev:demo` in the query field. Similarly, if you want logs with the tag `dev:prod` to trigger signals with a severity of `MEDIUM`, enter `dev:prod` in the query field. + +### Other parameters + +#### Enable optional group by {% #enable-group-by-historical-third-party %} + +{% partial file="security/cloud_siem/enable_group_by.mdoc.md" /%} + +#### Group signals {% #group-signals-historical-third-party %} + +{% partial file="security/cloud_siem/group_signals.mdoc.md" /%} +{% /if %} + + +{% if equals($cloud_siem_detection_rule_type, "scheduled_rule") %} +## Add custom schedule + +You can set specific evaluation time and how often it runs by creating a [custom schedule](#create-custom-schedule) or using a [recurrence rule (RRULE)](#use-rrule). + +### Create custom schedule + +{% img src="security/security_monitoring/detection_rules/custom_schedule.png" alt="The Use custom schedule section with an example" style="width:100%;" /%} + +1. Select **Create Custom Schedules**. +1. Set how often and at what time you want the rule to run. + +### Use RRULE + +{% img src="security/security_monitoring/detection_rules/rrule_example.png" alt="The Use RRULE section with an example" style="width:100%;" /%} + +Recurrence rule (RRULE) is a property name from the [iCalendar RFC][8], which is the standard for defining recurring events. Use the [official RRULE generator][9] to generate recurring rules. Leverage RRULEs to cover more advanced scheduling use cases. + +For example, if the RRULE is: + +```text +FREQ=DAILY;INTERVAL=1;BYHOUR=6;BYMINUTE=0 +``` + +The example RRULE runs the scheduled rule once a day at 6:00 AM. + +{% alert level="info" %} +- Attributes specifying the duration in RRULE are not supported (for example, `DTSTART`, `DTEND`, `DURATION`). +- Evaluation frequencies must be a day or longer. For shorter evaluation frequencies, use the default monitor schedules. +{% /alert %} + +To write a custom RRULE for your detection rule: + +1. Select ** Use RRULE**. +1. Set the date and time for when you want the rule to start. +1. Input a [RRULE string][9] to set how often you want the rule to run. +{% /if %} + + +{% if equals($cloud_siem_detection_rule_type, "historical_job") %} +## Notify when job is complete + +(Optional) Click **Add Recipient** to send notifications upon the completion of job analysis. See [Notification channels][3] for more information. +{% /if %} + +## Describe your playbook + +{% partial file="security/security-rule-say-whats-happening.mdoc.md" /%} + + +{% if or(equals($cloud_siem_detection_rule_type, "real_time_rule"),equals($cloud_siem_detection_rule_type, "scheduled_rule")) %} +## Create a suppression + +{% partial file="security/cloud_siem/create_suppression.mdoc.md" /%} +{% /if %} + +[1]: /logs/search_syntax/ +[2]: https://app.datadoghq.com/security/siem/rules/new +[3]: /security_platform/notifications/#notification-channels +[4]: /security/notifications/rules/ +[5]: /ddsql_reference/ +[6]: https://app.datadoghq.com/security/configuration/datasets +[7]: /logs/explorer/calculated_fields/ +[8]: https://icalendar.org/rrule-tool.html +[9]: https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html \ No newline at end of file diff --git a/customization_config/en/option_groups/cloud_siem_custom_detection_rules.yaml b/customization_config/en/option_groups/cloud_siem_custom_detection_rules.yaml new file mode 100644 index 00000000000..05f0b8f1f62 --- /dev/null +++ b/customization_config/en/option_groups/cloud_siem_custom_detection_rules.yaml @@ -0,0 +1,96 @@ +cloud_siem_detection_rule_type_options: + - id: real_time_rule + default: true + - id: scheduled_rule + - id: historical_job + +# cloud_siem_detection_threshold_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule +# - id: historical_job + +# cloud_siem_detection_new_value_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule +# - id: historical_job + +# cloud_siem_detection_anomaly_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule +# - id: historical_job + +# cloud_siem_detection_content_anomaly_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule +# - id: historical_job + +# cloud_siem_detection_impossible_travel_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule +# - id: historical_job + +# cloud_siem_detection_third_party_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule +# - id: historical_job + +# cloud_siem_detection_sequence_rule_type_options: +# - id: real_time_rule +# default: true + +# cloud_siem_detection_signal_correlation_rule_type_options: +# - id: real_time_rule +# default: true +# - id: scheduled_rule + +cloud_siem_detection_real_time_rule_search_query_options: + - id: threshold + default: true + - id: new_value + - id: anomaly + - id: content_anomaly + - id: impossible_travel + - id: third_party + - id: sequence + - id: signal_correlation + +cloud_siem_detection_scheduled_rule_search_query_options: + - id: threshold + default: true + - id: new_value + - id: anomaly + - id: content_anomaly + - id: impossible_travel + - id: third_party + - id: signal_correlation + +cloud_siem_detection_historical_job_search_query_options: + - id: threshold + default: true + - id: new_value + - id: anomaly + - id: content_anomaly + - id: impossible_travel + - id: third_party + +cloud_siem_detection_rule_query_language_options: + - id: event_query + default: true + - id: sql + +# cloud_siem_detection_rule_search_query_options: +# - id: threshold +# default: true +# - id: new_value +# - id: anomaly +# - id: content_anomaly +# - id: impossible_travel +# - id: third_party +# - id: signal_correlation +# - id: sequence \ No newline at end of file diff --git a/customization_config/en/options/general.yaml b/customization_config/en/options/general.yaml index c3685c0c95d..b1e1995972f 100644 --- a/customization_config/en/options/general.yaml +++ b/customization_config/en/options/general.yaml @@ -506,4 +506,40 @@ options: id: cpp - label: Elixir - id: elixir \ No newline at end of file + id: elixir + +- id: real_time_rule + label: Real-time rule + +- id: scheduled_rule + label: Scheduled rule + +- id: historical_job + label: Historical job + +- id: new_value + label: New value + +- id: anomaly + label: Anomaly + +- id: content_anomaly + label: Content anomaly + +- id: impossible_travel + label: Impossible travel + +- id: third_party + label: Third party + +- id: signal_correlation + label: Signal correlation + +- id: sequence + label: Sequence + +- id: event_query + label: Event Query + +- id: sql + label: SQL \ No newline at end of file diff --git a/customization_config/en/traits/general.yaml b/customization_config/en/traits/general.yaml index 6408fd9e077..93d788c9de7 100644 --- a/customization_config/en/traits/general.yaml +++ b/customization_config/en/traits/general.yaml @@ -51,4 +51,19 @@ traits: - id: protocol label: Protocol type: text - internal_notes: For example, HTTP, gRPC, or SOCKS. \ No newline at end of file + internal_notes: For example, HTTP, gRPC, or SOCKS. + +- id: cloud_siem_detection_rule_type + label: "Rule type" + type: text + internal_notes: The rule type a user wants to use when creating a custom detection rule for Cloud SIEM. + +- id: cloud_siem_detection_rule_search_query + label: "Search query" + type: text + internal_notes: The search query a user wants to use when creating a custom detection rule for Cloud SIEM. + +- id: cloud_siem_detection_rule_query_language + label: "Query language" + type: text + internal_notes: The search query language a user wants to use when creating a custom detection rule for Cloud SIEM. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/add_calculated_fields.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/add_calculated_fields.mdoc.md new file mode 100644 index 00000000000..64a26868371 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/add_calculated_fields.mdoc.md @@ -0,0 +1,7 @@ +1. Click **Add** and select **Calculated fields**. +1. In **Name your field**, enter a descriptive name that indicates the purpose of the calculated field. + - For example, if you want to combine users' first and last name into one field, you might name the calculated field `fullName`. +1. In the **Define your formula** field, enter a formula or expression, which determines the result to be computed and stored as the value of the calculated field for each log event. + - See [Calculated Fields Expressions Language][701] for information on syntax and language constructs. + +[701]: /logs/explorer/calculated_fields/expression_language/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/add_reference_tables.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/add_reference_tables.mdoc.md new file mode 100644 index 00000000000..456af4831c6 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/add_reference_tables.mdoc.md @@ -0,0 +1,5 @@ +1. Click the **Add** button next to the query editor and select **Join with Reference Table**. +1. In the **Inner join with reference table** dropdown menu, select your reference table. +1. In the **where field** dropdown menu, select the log field to join on. +1. Select the **IN** or **NOT IN** operator to filter in or filter out matching logs. +1. In the **column** dropdown menu, select the column of the reference table to join on. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/anomaly_query.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/anomaly_query.mdoc.md new file mode 100644 index 00000000000..3bcfe369467 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/anomaly_query.mdoc.md @@ -0,0 +1,6 @@ +1. (Optional) In the **Count** dropdown menu, select attributes whose unique values you want to count during the specified time frame. +1. (Optional) In the **group by** dropdown menu, select attributes you want to group by. + - The defined `group by` generates a signal for each `group by` value. + - Typically, the `group by` is an entity (like user or IP). The `group by` can also join the queries together. + - Joining logs that span a time frame can increase the confidence or severity of the security signal. For example, if you want to detect a successful brute force attack, both successful and unsuccessful authentication logs must be correlated for a user. + - Anomaly detection inspects how the `group by` attribute has behaved in the past. If a `group by` attribute is seen for the first time (for example, the first time an IP is communicating with your system) and is anomalous, it does not generate a security signal because the anomaly detection algorithm has no historical data to compare with. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/content_anomaly_options.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/content_anomaly_options.mdoc.md new file mode 100644 index 00000000000..e4f05ebc426 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/content_anomaly_options.mdoc.md @@ -0,0 +1,9 @@ +In the **Content anomaly detection options** section, specify the parameters to assess whether a log is anomalous or not. +- Content anomaly detection balances precision and sensitivity using several rule parameters that you can set: + 1. Similarity threshold: Defines how dissimilar a field value must be to be considered anomalous (default: `70%`). + 1. Minimum similar items: Sets how many similar historical logs must exist for a value to be considered normal (default: `1`). + 1. Evaluation window: The time frame during which anomalies are counted toward a signal (for example, a 10-minute time frame). +- These parameters help to identify field content that is both unusual and rare, filtering out minor or common variations. +- See [Anomaly detection parameters][601] for more information. + +[601]: /security/cloud_siem/detect_and_monitor/custom_detection_rules/content_anomaly/#anomaly-detection-parameters \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/content_anomaly_query.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/content_anomaly_query.mdoc.md new file mode 100644 index 00000000000..85b70cc1558 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/content_anomaly_query.mdoc.md @@ -0,0 +1,9 @@ +1. In the **Detect anomaly** field, specify the fields whose values you want to analyze. +1. In the **group by** field, specify the fields you want to group by. + - The defined `group by` generates a signal for each `group by` value. + - Typically, the `group by` is an entity (like user or IP). The `group by` can also join the queries together. + - Joining logs that span a time frame can increase the confidence or severity of the security signal. For example, to detect a successful brute force attack, both successful and unsuccessful authentication logs must be correlated for a user. +1. In the **Learn for** dropdown menu, select the number of days for the learning period. During the learning period, the rule sets a baseline of normal field values and does not generate any signals. + {% alert level="info" %} + If the detection rule is modified, the learning period restarts at day `0`. + {% /alert %} \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/create_suppression.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/create_suppression.mdoc.md new file mode 100644 index 00000000000..6730344c98b --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/create_suppression.mdoc.md @@ -0,0 +1,16 @@ +(Optional) Create a suppression or add the rule to an existing suppression to prevent a signal from getting generated in specific cases. For example, if a user `john.doe` is triggering a signal, but their actions are benign and you do not want signals triggered from this user, add the following query into the **Add a suppression query** field: `@user.username:john.doe`. + +### Create new suppression + +1. Enter a name for the suppression rule. +1. (Optional) Enter a description. +1. Enter a suppression query. +1. (Optional) Add a log exclusion query to exclude logs from being analyzed. These queries are based on **log attributes**. + {% alert level="info" %} + The legacy suppression was based on log exclusion queries, but it is now included in the suppression rule's **Add a suppression query** step. + {% /alert %} + +### Add to existing suppression + +1. Click **Add to Existing Suppression**. +1. Select an existing suppression in the dropdown menu. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_decrease_severity.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_decrease_severity.mdoc.md new file mode 100644 index 00000000000..b1a74c12216 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_decrease_severity.mdoc.md @@ -0,0 +1,3 @@ +Toggle **Decrease severity for non-production environments** if you want to prioritize production environment signals over non-production signals. +- The severity of signals in non-production environments are decreased by one level from what is defined by the rule case. +- The severity decrement is applied to signals with an environment tag starting with `staging`, `test`, or `dev`. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_group_by.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_group_by.mdoc.md new file mode 100644 index 00000000000..de2dc425349 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_group_by.mdoc.md @@ -0,0 +1 @@ +Toggle the **Enable Optional Group By** section if you want to group events even when values are missing. If there is a missing value, a sample value is generated so that the log does not get excluded. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_instantaneous_baseline.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_instantaneous_baseline.mdoc.md new file mode 100644 index 00000000000..56053329805 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/enable_instantaneous_baseline.mdoc.md @@ -0,0 +1 @@ +Toggle **Enable instantaneous baseline** if you want to build the baseline based on past events for the first event received. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/forget_value.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/forget_value.mdoc.md new file mode 100644 index 00000000000..dce4300332b --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/forget_value.mdoc.md @@ -0,0 +1 @@ +In the **Forget Value** dropdown, select the number of days (**1**-**30 days**) after which the value is forgotten. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/group_signals.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/group_signals.mdoc.md new file mode 100644 index 00000000000..63a64d73adb --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/group_signals.mdoc.md @@ -0,0 +1 @@ +Toggle **Group signals** if you want to reduce the number of signals generated. Then, select one or more groups for which you want to generate one security signal each. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/impossible_travel_query.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/impossible_travel_query.mdoc.md new file mode 100644 index 00000000000..9164a42e2a9 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/impossible_travel_query.mdoc.md @@ -0,0 +1,10 @@ +1. In the **User attribute** dropdown menu, select the log attribute that contains the user ID. This can be an identifier like an email address, user name, or account identifier. +1. The **Location attribute** value is automatically set to `@network.client.geoip`. + - The `location attribute` specifies which field holds the geographic information for a log. + - The only supported value is `@network.client.geoip`, which is enriched by the [GeoIP parser][801] to give a log location information based on the client's IP address. +1. Select the **Baseline user locations** checkbox if you want Datadog to learn regular access locations before triggering a signal. + - When selected, signals are suppressed for the first 24 hours. During that time, Datadog learns the user's regular access locations. This can be helpful to reduce noise and infer VPN usage or credentialed API access. + - See [How the impossible detection method works][802] for more information. + +[801]: /logs/log_configuration/processors/?tab=ui#geoip-parser +[802]: /security/cloud_siem/detect_and_monitor/custom_detection_rules/impossible_travel/#how-the-impossible-travel-method-works \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/job_multi_triggering.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/job_multi_triggering.mdoc.md new file mode 100644 index 00000000000..d2e3674dab9 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/job_multi_triggering.mdoc.md @@ -0,0 +1,7 @@ +In the **Job multi-triggering behavior** section, configure how often to keep updating the same signal when new values are detected within a specified time frame. For example, the same signal updates when any new value is detected within 1 hour, for a maximum duration of 24 hours. +- An `evaluation window` defines a sliding period in which at least one case evaluates as true and assesses cases in real time. +- After a signal is generated, the signal remains "open" if a case is matched at least once within the `keep alive` window. Each time a new event matches any of the cases, the *last updated* timestamp is updated for the signal. +- A signal closes after the time exceeds the `maximum signal duration`, regardless of the query being matched. This time is calculated from the first seen timestamp. +{% alert level="info" %} +The `evaluation window` must be less than or equal to the `keep alive` and `maximum signal duration`. +{% /alert %} \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/new_value_query.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/new_value_query.mdoc.md new file mode 100644 index 00000000000..81e39bc0142 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/new_value_query.mdoc.md @@ -0,0 +1,16 @@ +1. In the **Detect new value** dropdown menu, select the attributes you want to detect. + - For example, you can create a query for successful user authentication with the following settings: + - **Detect new value** is `country` + - **group by** is `user` + - Learning duration is `after 7 days` + {% br /%}Then, logs coming in over the next 7 days are evaluated with those configured values. If a log comes in with a new value after the learning duration (`7 days`), a signal is generated, and the new value is learned to prevent future signals with this value. + - You can also identify users and entities using multiple **Detect new value** attributes in a single query. + - For example, if you want to detect when a user signs in from a new device and from a country that they've never signed in from before, add `device_id` and `country_name` to the **Detect new value** field. +1. (Optional) Define a signal grouping in the **group by** dropdown menu. + - The defined `group by` generates a signal for each `group by` value. + - Typically, the `group by` is an entity (like user or IP address). +1. In the dropdown menu to the right of **group by**, select the learning duration. +1. (Optional) Define a signal grouping in the **group by** dropdown menu. + - The defined `group by` generates a signal for each `group by` value. + - Typically, the `group by` is an entity (like user or IP address). +1. In the dropdown menu to the right of **group by**, select the learning duration. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/rule_multi_triggering.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/rule_multi_triggering.mdoc.md new file mode 100644 index 00000000000..02c4657162f --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/rule_multi_triggering.mdoc.md @@ -0,0 +1,7 @@ +Configure how often you want to keep updating the same signal if new values are detected within a specified time frame. For example, the same signal updates if any new value is detected within 1 hour, for a maximum duration of 24 hours. +- An `evaluation window` is specified to match when at least one of the cases matches true. This is a sliding window and evaluates cases in real time. +- After a signal is generated, the signal remains "open" if a case is matched at least once within the `keep alive` window. Each time a new event matches any of the cases, the *last updated* timestamp is updated for the signal. +- A signal closes after the time exceeds the `maximum signal duration`, regardless of the query being matched. This time is calculated from the first seen timestamp. +{% alert level="info" %} +The `evaluation window` must be less than or equal to the `keep alive` and `maximum signal duration`. +{% /alert %} \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/rule_multi_triggering_content_anomaly.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/rule_multi_triggering_content_anomaly.mdoc.md new file mode 100644 index 00000000000..a21207effea --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/rule_multi_triggering_content_anomaly.mdoc.md @@ -0,0 +1,6 @@ +Configure how often you want to keep updating the same signal if new values are detected within a specified time frame. For example, the same signal updates if any new value is detected within 1 hour, for a maximum duration of 24 hours. +- After a signal is generated, the signal remains "open" if a case is matched at least once within the `keep alive` window. Each time a new event matches any of the cases, the *last updated* timestamp is updated for the signal. +- A signal closes after the time exceeds the `maximum signal duration`, regardless of the query being matched. This time is calculated from the first seen timestamp. +{% alert level="info" %} +The `evaluation window` must be less than or equal to the `keep alive` and `maximum signal duration`. +{% /alert %} \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_anomaly.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_anomaly.mdoc.md new file mode 100644 index 00000000000..e47b7519758 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_anomaly.mdoc.md @@ -0,0 +1,8 @@ +1. In the **Set severity to** dropdown menu, select the appropriate severity level (`INFO`, `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`). +1. In the **Anomaly Percentile** dropdown menu, select a minimum percentage required for Cloud SIEM to generate a signal.{% br /%}The anomaly percentile refers to the log volume over the selected time period to your historical log volumes. If you select 99.5%, then Cloud SIEM only generates a signal if the number of logs is greater than 99.5% of all prior periods. +1. (Optional) In the **And notify** section, click **Add Recipient** to configure [notification targets][101]. + - You can create [notification rules][102] to manage notifications automatically, avoiding manual edits for each detection rule. + + +[101]: /security_platform/notifications/#notification-channels +[102]: /security/notifications/rules/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_content_anomaly.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_content_anomaly.mdoc.md new file mode 100644 index 00000000000..dcbca156ae4 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_content_anomaly.mdoc.md @@ -0,0 +1,17 @@ +1. (Optional) Click the pencil icon next to **Condition 1** if you want to rename the condition. This name is appended to the rule name when a signal is generated. +1. In the **Set severity to** dropdown menu, select the appropriate severity level (`INFO`, `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`). +1. In the **Anomaly count** field, enter the condition for how many anomalous logs within the specified window are required to trigger a signal. + - For example, if the condition is `a >= 3` where `a` is the query, a signal is triggered if there are at least three anomalous logs within the evaluation window. + - All rule conditions are evaluated as condition statements. Thus, the order of the conditions affects which notifications are sent because the first condition to match generates the signal. Click and drag your rule conditions to change their ordering. + - A rule condition contains logical operations (`>`, `>=`, `&&`, `||`) to determine if a signal should be generated based on the event counts in the previously defined queries. + - The ASCII lowercase query labels are referenced in this section. An example rule condition for query `a` is `a > 3`. + {% alert level="info" %} + The query label must precede the operator. For example, `a > 3` is allowed; `3 < a` is not allowed. + {% /alert %} +1. In the **within a window of** dropdown menu, select the time period during which a signal is triggered if the condition is met. + - An `evaluation window` is specified to match when at least one of the cases matches true. This is a sliding window and evaluates cases in real time. +1. In the **And notify** section, click **Add Recipient** to optionally configure [notification targets][101]. + - You can also create [notification rules][102] to avoid manual edits to notification preferences for individual detection rules. + +[101]: /security_platform/notifications/#notification-channels +[102]: /security/notifications/rules/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_severity_notify_only.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_severity_notify_only.mdoc.md new file mode 100644 index 00000000000..6aa498b0b70 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_severity_notify_only.mdoc.md @@ -0,0 +1,7 @@ +1. In the **Set severity to** dropdown menu, select the appropriate severity level (`INFO`, `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`). +1. (Optional) In the **And notify** section, click **Add Recipient** to configure [notification targets][101]. + - You can create [notification rules][102] to manage notifications automatically, avoiding manual edits for each detection rule. + + +[101]: /security_platform/notifications/#notification-channels +[102]: /security/notifications/rules/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_then_operator.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_then_operator.mdoc.md new file mode 100644 index 00000000000..f0532d5b422 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_then_operator.mdoc.md @@ -0,0 +1,19 @@ +1. If you want to create a simple condition, leave the selection as is. If you want to create a `then` condition, click **THEN condition**. + - Use the **Then condition** when you want to trigger a signal if query A occurs and then query B occurs. + {% alert level="info" %} + The `then` operator can only be used on a single rule condition. + {% /alert %} +1. (Optional) Click the pencil icon next to **Condition 1** if you want to rename the condition. This name is appended to the rule name when a signal is generated. +1. In the **Set severity to** dropdown menu, select the appropriate severity level (`INFO`, `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`). +1. If you are creating a **Simple condition**, enter the condition when a signal should be created. If you are creating a **Then condition**, enter the conditions required for a signal to be generated. + - All rule conditions are evaluated as conditional statements. Thus, the order of the conditions affects which notifications are sent because the first condition to match generates the signal. Click and drag your rule conditions to change their order. + - A rule condition contains logical operations (`>`, `>=`, `<`, `&&`, `||`) to determine if a signal should be generated based on the event counts in the previously defined queries. + - The ASCII lowercase query labels are referenced in this section. An example rule condition for query `a` is `a > 3`. + {% alert level="info" %} + The query label must precede the operator. For example, `a > 3` is allowed; `3 < a` is not allowed. + {% /alert %} +1. In the **And notify** section, click **Add Recipient** to optionally configure [notification targets][101]. + - You can create [notification rules][102] to manage notifications automatically, avoiding manual edits for each detection rule. + +[101]: /security_platform/notifications/#notification-channels +[102]: /security/notifications/rules/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_third_party.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_third_party.mdoc.md new file mode 100644 index 00000000000..0d1f0cd0f49 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_third_party.mdoc.md @@ -0,0 +1,11 @@ +1. (Optional) Click the pencil icon next to **Condition 1** if you want to rename the condition. This name is appended to the rule name when a signal is generated. +1. In the **Set severity to** dropdown menu, select the appropriate severity level (`INFO`, `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`). +1. In the **Query** field, enter the tags of a log that you want to trigger a signal. + - For example, if you want logs with the tag `dev:demo` to trigger signals with a severity of `INFO`, enter `dev:demo` in the query field. Similarly, if you want logs with the tag `dev:prod` to trigger signals with a severity of `MEDIUM`, enter `dev:prod` in the query field. +1. (Optional) In the **And notify** section, click **Add Recipient** to configure [notification targets][101]. + - You can also create [notification rules][102] to avoid manual edits to notification preferences for individual detection rules. +1. For the `else` condition, follow steps 3 and 4. + - The `else` condition is the default condition. If you don't add any other conditions, then all logs trigger a signal with the severity set in the default condition. + +[101]: /security_platform/notifications/#notification-channels +[102]: /security/notifications/rules/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_threshold.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_threshold.mdoc.md new file mode 100644 index 00000000000..e9d7bb392b4 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/set_conditions_threshold.mdoc.md @@ -0,0 +1,21 @@ +1. If you have a single query, skip to step 2. If you have multiple queries, you can create a **Simple condition** or **Then condition**. + - If you want to create a **simple** condition, leave the selection as is. + - If you want to create a **then** condition, click **THEN condition**. + - Use the **Then condition** when you want to trigger a signal if query A occurs and then query B occurs. + {% alert level="info" %} + The `then` operator can only be used on a single rule condition. + {% /alert %} +1. (Optional) Click the pencil icon next to **Condition 1** if you want to rename the condition. This name is appended to the rule name when a signal is generated. +1. In the **Set severity to** dropdown menu, select the appropriate severity level (`INFO`, `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`). +1. If you are creating a **Simple condition**, enter the condition when a signal should be created. If you are creating a **Then condition**, enter the conditions required for a signal to be generated. + - All rule conditions are evaluated as condition statements. Thus, the order of the conditions affects which notifications are sent because the first condition to match generates the signal. Click and drag your rule conditions to change their order. + - A rule condition contains logical operations (`>`, `>=`, `<`, `&&`, `||`) to determine if a signal should be generated based on the event counts in the previously defined queries. + - The ASCII lowercase query labels are referenced in this section. An example rule condition for query `a` is `a > 3`. + {% alert level="info" %} + The query label must precede the operator. For example, `a > 3` is allowed; `3 < a` is not allowed. + {% /alert %} +1. (Optional) In the **And notify** section, click **Add Recipient** to configure [notification targets][101]. + - You can also create [notification rules][102] to avoid manual edits to notification preferences for individual detection rules. + +[101]: /security_platform/notifications/#notification-channels +[102]: /security/notifications/rules/ \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/threshold_query.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/threshold_query.mdoc.md new file mode 100644 index 00000000000..443d9fddb6b --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/threshold_query.mdoc.md @@ -0,0 +1,5 @@ +1. (Optional) In the **Count** dropdown menu, select attributes whose unique values are counted over the specified time frame. +1. (Optional) In the **group by** dropdown menu, select attributes you want to group by. + - The defined `group by` generates a signal for each `group by` value. + - Typically, the `group by` is an entity (like user, or IP). The `group by` is also used to join the queries together. + - Joining logs that span a time frame can increase the confidence or severity of the security signal. For example, to detect a successful brute force attack, both successful and unsuccessful authentication logs must be correlated for a user. \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/cloud_siem/unit_testing.mdoc.md b/layouts/shortcodes/mdoc/en/security/cloud_siem/unit_testing.mdoc.md new file mode 100644 index 00000000000..6697e013363 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/cloud_siem/unit_testing.mdoc.md @@ -0,0 +1,10 @@ +1. To construct a sample log, you can: + 1. Navigate to [Log Explorer][601] in a new window. + 1. In the search bar, enter the query you are using for the detection rule. + 1. Select one of the logs. + 1. Click the export button at the top right side of the log side panel, and then select **Copy**. +1. Navigate back to the **Unit Test** modal, and then paste the log into the text box. Edit the sample as needed for your use case. +1. Toggle the switch for **Query is expected to match based on the example event** to fit your use case. +1. Click **Run Query Test**. + +[601]: https://app.datadoghq.com/logs \ No newline at end of file diff --git a/layouts/shortcodes/mdoc/en/security/security-rule-say-whats-happening.mdoc.md b/layouts/shortcodes/mdoc/en/security/security-rule-say-whats-happening.mdoc.md new file mode 100644 index 00000000000..55204b52872 --- /dev/null +++ b/layouts/shortcodes/mdoc/en/security/security-rule-say-whats-happening.mdoc.md @@ -0,0 +1,11 @@ +1. Enter a **Rule name**. The name appears in the detection rules list view and the title of the security signal. +1. In the **Rule message** section, use [notification variables][201] and Markdown to customize the notifications sent when a signal is generated. + - You can use [template variables][202] in the notification to inject dynamic context from triggered logs directly into a security signal and its associated notifications. + - See the [Notification Variables documentation][201] for more information and examples. +1. Use the **Tag resulting signals** dropdown menu to add tags to your signals. For example, `security:attack` or `technique:T1110-brute-force`. + {% alert level="info" %} + The tag `security` is special. This tag is used to classify the security signal. The recommended options are `attack`, `threat-intel`, `compliance`, `anomaly`, and `data-leak`. + {% /alert %} + +[201]: /security_platform/notifications/variables/ +[202]: /security_platform/notifications/variables/#template-variables \ No newline at end of file diff --git a/static/images/security/security_monitoring/detection_rules/anomaly_notification.png b/static/images/security/security_monitoring/detection_rules/anomaly_notification.png new file mode 100644 index 00000000000..63380a6cd81 Binary files /dev/null and b/static/images/security/security_monitoring/detection_rules/anomaly_notification.png differ