Working with alerts

Use alerting to keep track of the different activities that take place on the system. The alerting ecosystem consists of the monitor, trigger, action, and channels.

Viewing alerts

Generated alerts are displayed on the Audit Store Dashboards. View and acknowledge the alerts from the alerting dashboard by navigating to OpenSearch Plugins > Alerting > Alerts. The alerting dashboard is shown in the following figure.

Destinations for alerts are moved to channels in Notifications. For more information about working with Monitors, Alerts, and Notifications, refer to the section Monitors in https://opensearch.org/docs/latest/dashboards/.

Creating notifications

Create notification channels to receive alerts as per individual requirements. The alerts are sent to the destination specified in the channel.

Creating a custom webhook notification

A webhook notification sends the alerts generated by a monitor to a destination, such as, a web page.

Perform the following steps to configure the notification channel for generating webhook alerts:

  1. Log in to the ESA Web UI.

  2. Navigate to Audit Store > Dashboard.

    The Audit Store Dashboards appears. If a new tab does not automatically open, click Open in a new tab.

  3. From the menu, navigate to Management > Notifications > Channels.

  4. Click Create channel.

  5. Specify the following information under Name and description.

    • Name: Http_webhook
    • Description: For generating http webhook alerts.
  6. Specify the following information under Configurations.

  7. Click Send test message to send a message to the email recipients.

  8. Click Create to create the channel.

    The webhook is set up successfully.

  9. Proceed to create a monitor and attach the channel created using the steps from Creating the monitor.

Creating email alerts using custom webhook

An email notification sends alerts generated by a monitor to an email address. It is also possible to configure the SMTP channel for sending an email alert. It is recommended to send email alerts using custom webhooks, which offers added security. The email alerts can be encrypted or non-encrypted. Accordingly, the required SMTP settings for email notifications must be configured on the ESA.

Perform the following steps to configure the notification channel for generating email alerts using custom webhooks:

Ensure that the following is configured as per the requirement:

  • Configuring SMTP on the ESA, refer here.
  1. Log in to the ESA Web UI.

  2. Navigate to Audit Store > Dashboard.

    The Audit Store Dashboards appears. If a new tab does not automatically open, click Open in a new tab.

  3. From the menu, navigate to OpenSearch Plugins > Notifications > Channels.

  4. Click Create channel.

  5. Specify the following information under Name and description.

    • Name: Unsecure_smtp_email
    • Description: For generating unsecured SMTP email alerts.
  6. Specify the following information under Configurations.

    • Channel type: Custom webhook
    • Define endpoints by: Custom attributes URL
    • Type: HTTP
    • Host: <ESA_IP>
    • Port: 8588
    • Path: rest/alerts/alerts/send_smtp_email_alerts
  7. Under Query parameters, click Add parameter and specify the following information. Click Add parameter and add cc and bcc, if required.

    • Key: to
    • Value: <email_ID>
  8. Under Webhook headers, click Add header and specify the following information.

    • Key: Pty-Username
    • Value: %internal_scheduler;
  9. Under Webhook headers, click Add header and specify the following information.

    • Key: Pty-Roles
    • Value: auditstore_admin
  10. Click Create to save the channel configuration.

    CAUTION: Do not click Send test message because the configuration for the channel is not complete.

    The success message appears and the channel is created. The webhook for the email alerts is set up successfully.

  11. Proceed to create a monitor and attach the channel created using the steps from Creating the monitor.

Perform the following steps to configure the notification channel for generating secure email alerts using custom webhooks:

Ensure that the following is configured as per the requirement:

  • Configuring SMTP on the ESA, refer here.
  1. Configure the certificates, if not already configured.

    1. Download the CA certificate of your SMTP server.

    2. Log in to the ESA Web UI.

    3. Upload the SMTP CA certificate on the ESA.

      1. Navigate to Settings > Network > Certificate Repository.

      2. Upload your CA certificate to the ESA.

      3. Select and activate your certificates in Management & Web Services from Settings > Network > Manage Certificates. For more information about ESA certificates, refer here.

    4. Update the smtp_config.json configuration file.

      1. Navigate to Settings > System > Files > smtp_config.json.

      2. Click the Edit the product file () icon.

      3. Update the following SMTP settings and the certificate information in the file. Sample values are provided in the following code, ensure that you use values as per individual requirements.

        • Set enabled to true to enable SMTP settings.

          "enabled": true, 
          
        • Specify the host address for the SMTP connection.

          "host": "192.168.1.10", 
          
        • Specify the port for the SMTP connection.

          "port": "25", 
          
        • Specify the email address of the sender for the SMTP connection.

          "sender_email_address": "<Email_ID>", 
          
        • Enable STARTTLS.

          "use_start_tls": "true", 
          
        • Enable server certificate validation.

          "verify_server_cert": "true", 
          
        • Specify the location for the CA certificate.

          "ca_file_path": "/etc/ksa/certificates/mng/CA.pem", 
          
      4. Click Save.

    5. Repeat the steps on the remaining nodes of the Audit Store cluster.

  2. Navigate to Audit Store > Dashboard.

    The Audit Store Dashboards appears. If a new tab does not automatically open, click Open in a new tab.

  3. From the menu, navigate to OpenSearch Plugins > Notifications > Channels.

  4. Click Create channel.

  5. Specify the following information under Name and description.

    • Name: Secure_smtp_email
    • Description: For generating secured SMTP email alerts.
  6. Specify the following information under Configurations.

    • Channel type: Custom webhook
    • Define endpoints by: Custom attributes URL
    • Type: HTTP
    • Host: <ESA_IP>
    • Port: 8588
    • Path: rest/alerts/alerts/send_secure_smtp_email_alerts
  7. Under Query parameters, click Add parameter and specify the following information. Click Add parameter and add cc and bcc, if required.

    • Key: to
    • Value: <email_ID>
  8. Under Webhook headers, click Add header and specify the following information.

    • Key: Pty-Username
    • Value: %internal_scheduler;
  9. Under Webhook headers, click Add header and specify the following information.

    • Key: Pty-Roles
    • Value: auditstore_admin
  10. Click Create to save the channel configuration.

    CAUTION: Do not click Send test message because the configuration for the channel is not complete.

    The success message appears and the channel is created. The webhook for the email alerts is set up successfully.

  11. Proceed to create a monitor and attach the channel created using the steps from Creating the monitor.

Creating an email notification

Perform the following steps to configure the notification channel for generating email alerts:

  1. Log in to the ESA Web UI.

  2. Navigate to Audit Store > Dashboard.

    The Audit Store Dashboards appears. If a new tab does not automatically open, click Open in a new tab.

  3. From the menu, navigate to Management > Notifications > Channels.

  4. Click Create channel.

  5. Specify the following information under Name and description.

    • Name: Email_alert
    • Description: For generating email alerts.
  6. Specify the following information under Configurations.

    • Channel type: Email
    • Sender type: SMTP sender
    • Default recipients: Specify the list of email addresses for receiving the alerts.
  7. Click Create SMTP sender and add the following parameters.

    • Sender name: Specify a descriptive name for sender.
    • Email address: Specify the email address that must receive the alerts.
    • Host: Specify the host name of the email server.
    • Port: 25
    • Encryption method: None
  8. Click Create.

  9. Click Send test message to send a message to the email recipients.

  10. Click Create to create the channel.

    The email alert is set up successfully.

  11. Proceed to create a monitor and attach the channel created using the steps from Creating the monitor.

Creating the monitor

A monitor tracks the system and sends an alert when a trigger is activated. Triggers cause actions to occur when certain criteria are met. Those criteria are set when a trigger is created. For more information about monitors, actions, and triggers, refer to Alerting.

Perform the following steps to create a monitor. The configuration specified here is just an example. For real use, create whatever configuration is needed, per individual requirements:

  1. Ensure that a notification is created using the steps from Creating notifications.

  2. From the menu, navigate to OpenSearch Plugins > Alerting > Monitors.

  3. Click Create Monitor.

  4. Specify a name for the monitor.

  5. For the Monitor defining method, select Extraction query editor.

  6. For the Schedule, select 30 Minutes.

  7. For the Index, select the required index.

  8. Specify the following query for the monitor. Modify the query as per the requirement.

    {
        "size": 0,
        "query": {
            "match_all": {
                "boost": 1
            }
        }
    }
    
  9. Click Add trigger and specify the information provided here.

    1. Specify a trigger name.

    2. Specify a severity level.

    3. Specify the following code for the trigger condition:

      ctx.results[0].hits.total.value > 0
      
  10. Click Add action.

  11. From the Channels list, select the required channel.

  12. Add the following code in the Message field. The default message displayed might not be formatted properly. Update the message by replacing the Line spaces with the n escape code. The message value is a JSON value, use escape characters to structure the email properly using valid JSON syntax.

```
{
"message": "Please investigate the issue.\n  - Trigger: {{ctx.trigger.name}}\n  - Severity: {{ctx.trigger.severity}}\n  - Period start: {{ctx.periodStart}}\n  - Period end: {{ctx.periodEnd}}",
"subject": "Monitor {{ctx.monitor.name}} just entered alert status"
}
```
>   The **message** value is a JSON value. Be sure to use escape characters to structure the email properly using valid JSON syntax. The default message displayed might not be formatted properly. Update the message by replacing the Line spaces with the **\\n** escape code.
  1. Select the Preview message check box to view the formatted email message.
  2. Click Send test message and verify the recipient’s inbox for the message.
  3. Click Save to update the configuration.
Last modified February 7, 2025