Working with alerts

Use alerting to keep track of the different activities that take place on the system. The alerting ecosystem consists of the monitor, trigger, action, and channels.

Viewing alerts

The alerts generated are displayed on the Audit Store Dashboards. View and acknowledge the alerts from the alerting dashboard by navigating to OpenSearch Plugins > Alerting > Alerts. The alerting dashboard is shown in the following figure.

Destinations for alerting are moved to channels in Notifications. For more information about working with Monitors, Alerts, and Notifications, refer to the section Monitors in https://opensearch.org/docs/2.15/dashboards/index/.

Creating a custom webhook for generating email alerts

Configure the SMTP channel for sending an email alert. Email alerts can only be sent using custom webhooks, which is supported. The email alerts can be encrypted or non-encrypted. Accordingly, the required SMTP settings for email notifications must be configured on the ESA.

Ensure that the following is configured as per the requirement:

  • Configuring SMTP on the ESA, refer here.
  • Configuring certificates for email, refer here.

Perform the following steps to configure the custom webhook for generating email alerts:

  1. Create a notification channel.

    1. Log in to the ESA Web UI.

    2. Navigate to Audit Store > Dashboard.

      The Audit Store Dashboards appears. If a new tab does not automatically open, click Open in a new tab.

    3. From the menu, navigate to OpenSearch Plugins > Notifications > Channels.

    4. Click Create channel.

    5. Specify the following information under Name and description.

      • Name: pty_destination
      • Description: For alerting email.
    6. Specify the following information under Configurations.

      • Channel type: Custom webhook
      • Define endpoints by: Custom attributes URL
      • Type: HTTP
      • Host: <ESA_IP>
      • Port: 8588
      • Path: rest/alerts/alerts/send_email_alerts
    7. Under Query parameters, click Add parameter and specify the following information. Click Add parameter and add cc and bcc, if required.

      • Key: to
      • Value: <email_ID>
    8. Under Webhook headers, click Add header and specify the following information.

      • Key: Pty-Username
      • Value: %internal_scheduler;
    9. Under Webhook headers, click Add header and specify the following information.

      • Key: Pty-Roles
      • Value: auditstore_admin
    10. Click Create to save the channel configuration.

      CAUTION: Do not click Sent test message because the configuration for the channel is not complete. The success message appears and the channel is created.

  2. Create a monitor for the pty_destination channel. The configurations specified here are for the example. Use the configurations as per the requirements.

    1. From the menu, navigate to OpenSearch Plugins > Alerting > Monitors.

    2. Click Create Monitor.

    3. Specify a name for the monitor.

    4. For the Monitor defining method, select Extraction query editor.

    5. For the Schedule, select 30 Minutes.

    6. For the Index, select pty_insight_analytics_audit_9.2.

    7. Specify the following query for the monitor. Modify the query as per the requirement.

      {
          "size": 0,
          "query": {
              "match_all": {
                  "boost": 1
              }
          }
      }
      
    8. Click Add trigger and specify the information provided here.

      1. Specify a trigger name.

      2. Specify a severity level.

      3. Specify the following code for the trigger condition:

        ctx.results[0].hits.total.value > 0
        
      4. Click Add action.

      5. From the Channels list, select pty_destination.

      6. Add the following code in the Message field. The default message displayed might not be formatted properly. Update the message by replacing the Line spaces with the n escape code. The message value is a JSON value, use escape characters to structure the email properly using valid JSON syntax.

      {
      "message": "Please investigate the issue.\n  - Trigger: {{ctx.trigger.name}}\n  - Severity: {{ctx.trigger.severity}}\n  - Period start: {{ctx.periodStart}}\n  - Period end: {{ctx.periodEnd}}",
      "subject": "Monitor {{ctx.monitor.name}} just entered alert status"
      }
      

      The message value is a JSON value, ensure that you use escape characters to structure the email properly using valid JSON syntax. The default message displayed might not be formatted properly. Update the message by replacing the Line spaces with the \n escape code.

      1. Select the Preview message check box to view the formatted email message.
      2. Click Send test message and verify the recipient’s inbox for the message.
    9. Click Save to update the configuration.

The webhook for the email alerts is set up successfully.